publications
Publications in reversed chronological order.
2024
-
Simon Seibt, Thomas Chang, Bartosz Von Rymon Lipinski, and Marc Erich LatoschikIn Eurographics 2024 - Posters, 2024This paper presents advancements in novel-view synthesis with 3D Gaussian Splatting (3DGS) using a dense and accurate SfM point cloud initialization approach. We address the challenge of achieving photorealistic renderings from sparse image data, where basic 3DGS training may result in suboptimal convergence, thus leading to visual artifacts. The proposed method enhances precision and density of initially reconstructed point clouds by refining 3D positions and extrapolating additional points, even for difficult image regions, e.g. with repeating patterns and suboptimal visual coverage. Our contributions focus on improving “Dense Feature Matching for Structure-from-Motion” (DFM4SfM) based on a homographic decomposition of the image space to support 3DGS training: First, a grid-based feature detection method is introduced for DFM4SfM to ensure a well-distributed 3D Gaussian initialization uniformly over all depth planes. Second, the SfM feature matching is complemented by a geometric plausibility check, priming the homography estimation and thereby improving the initial placement of 3D Gaussians. Experimental results on the NeRF-LLFF dataset demonstrate that this approach achieves superior qualitative and quantitative results, even for fewer views, and the potential for a significantly accelerated 3DGS training with faster convergence.
@inproceedings{Seibt2024DGSI, author = {Seibt, Simon and Chang, Thomas and Von Rymon Lipinski, Bartosz and Latoschik, Marc Erich}, title = {Dense 3D Gaussian Splatting Initialization for Sparse Image Data}, booktitle = {Eurographics 2024 - Posters}, year = {2024}, } -
Thomas Chang, Simon Seibt, and Bartosz Von Rymon LipinskiIn IEEE Conference on Computer Vision and Pattern Recognition, 2024Superpixels play a crucial role in image processing by partitioning an image into clusters of pixels with similar visual attributes. This facilitates subsequent image processing tasks, offering computational advantages over the manipulation of individual pixels. While numerous oversegmentation techniques have emerged in recent years, many rely on predefined initialization and termination criteria. In this paper, a novel top-down superpixel segmentation algorithm called Hierarchical Histogram Threshold Segmentation (HHTS) is introduced. It eliminates the need for initialization and implements auto-termination, outperforming state-of-the-art methods w.r.t. boundary recall. This is achieved by iteratively partitioning individual pixel segments into foreground and background and applying intensity thresholding across multiple color channels. The underlying iterative process constructs a superpixel hierarchy that adapts to local detail distributions until color information exhaustion. Experimental results demonstrate the superiority of the proposed approach in terms of boundary adherence, while maintaining competitive runtime performance on the BSDS500 and NYUV2 datasets. Furthermore, an application of HHTS in refining machine learning-based semantic segmentation masks produced by the Segment Anything Foundation Model (SAM) is presented.
@inproceedings{Chang2024HHTS, author = {Chang, Thomas and Seibt, Simon and Von Rymon Lipinski, Bartosz}, title = {Hierarchical Histogram Threshold Segmentation - Auto-terminating High-detail Oversegmentation}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition}, year = {2024}, }
2023
-
Simon Seibt, Bartosz Von Rymon Lipinski, Thomas Chang, and Marc Erich LatoschikIn IEEE International Conference on Image Processing Workshops, 2023Structure from motion (SfM) is a fundamental task in computer vision and allows recovering the 3D structure of a stationary scene from an image set. Finding robust and accurate feature matches plays a crucial role in the early stages of SfM. So in this work, we propose a novel method for computing image correspondences based on dense feature matching (DFM) using homographic decomposition: The underlying pipeline provides refinement of existing matches through iterative rematching, detection of occlusions and extrapolation of additional matches in critical image areas between image pairs. Our main contributions are improvements of DFM specifically for SfM, resulting in global refinement and global extrapolation of image correspondences between related views. Furthermore, we propose an iterative version of the Delaunay-triangulation-based outlier detection algorithm for robust processing of repeated image patterns. Through experiments, we demonstrate that the proposed method significantly improves the reconstruction accuracy.
@inproceedings{Seibt2023DFM4SfM, author = {Seibt, Simon and Von Rymon Lipinski, Bartosz and Chang, Thomas and Latoschik, Marc Erich}, title = {DFM4SfM - Dense Feature Matching for Structure from Motion}, booktitle = {IEEE International Conference on Image Processing Workshops}, year = {2023}, } -
Simon Seibt, Michael Arold, Bartosz Von Rymon Lipinski, Uwe Wienkop, and Marc Erich LatoschikIn Pattern Recognition (Proceedings of DAGM GCPR), 2023Image stitching plays a crucial role for various computer vision applications, like panoramic photography, video production, medical imaging and satellite imagery. It makes it possible to align two images captured at different views onto a single image with a wider field of view. However, for 3D scenes with high depth complexity and images captured from two different positions, the resulting image pair may exhibit significant parallaxes. Stitching images with multiple or large apparent motion shifts remains a challenging task, and existing methods often fail in such cases. In this paper, a novel image stitching pipeline is introduced, addressing the aforementioned challenge: First, iterative dense feature matching is performed, which results in a multi-homography decomposition. Then, this output is used to compute a per-pixel multidimensional weight map of the estimated homographies for image alignment via weighted warping. Additionally, the homographic image space decomposition is exploited using combinatorial analysis to identify parallaxes, resulting in a parallax-aware overlapping region: Parallax-free overlapping areas only require weighted warping and blending. For parallax areas, these operations are omitted to avoid ghosting artifacts. Instead, histogram- and mask-based color mapping is performed to ensure visual color consistency. The presented experiments demonstrate that the proposed method provides superior results regarding precision and handling of parallaxes.
@inproceedings{Seibt2023MHS, author = {Seibt, Simon and Arold, Michael and Von Rymon Lipinski, Bartosz and Wienkop, Uwe and Latoschik, Marc Erich}, title = {Parallax-aware Image Stitching based on Homographic Decomposition}, booktitle = {Pattern Recognition (Proceedings of DAGM GCPR)}, year = {2023}, publisher = {Springer International Publishing}, } -
Simon Seibt, Bastian Kuth, Bartosz Von Rymon Lipinski, Thomas Chang, and Marc Erich LatoschikVirtual Reality & Intelligent Hardware (Proceedings of CGI), 2023The demand for interactive photorealistic 3D environments has increased in recent years and in various fields such as architecture, engineering and entertainment. Nevertheless, achieving a balance between quality and performance for high-performance 3D applications and Virtual Reality (VR) remains a challenge. This paper addresses this issue by revisiting and extending view interpolation for image-based rendering, enabling the exploration of spacious open environments in 3D and VR. Therefore, we introduce multi-morphing, a novel rendering method based on a spatial data structure of 2D image patches, called the image graph. With this approach, novel views can be rendered with up to six degrees of freedom using only a sparse set of views. The rendering process does not require 3D reconstruction of geometry, nor per-pixel depth information: All relevant data for output is extracted from local morphing cells of the image graph. Detection of parallax image regions during preprocessing reduces rendering artifacts by extrapolating image patches from adjacent cells in real-time. Additionally, a GPU-based solution to resolve exposure inconsistencies within a dataset is presented, enabling seamless transitions of brightness when moving between areas with varying light intensities. Experiments on multiple real-world and synthetic scenes demonstrate that the presented method achieves high “VR-compatible” frame rates, even on mid-range and legacy hardware, respectively.
2022
-
Simon Seibt, Bartosz Von Rymon Lipinski, and Marc Erich LatoschikIEEE Access, 2022Finding robust and accurate feature matches is a fundamental problem in computer vision. However, incorrect correspondences and suboptimal matching accuracies lead to significant challenges for many real-world applications. In conventional feature matching, corresponding features in an image pair are greedily searched using their descriptor distance. The resulting matching set is then typically used as input for geometric model fitting methods to find an appropriate fundamental matrix and filter out incorrect matches. Unfortunately, this basic approach cannot solve all practical problems, such as fundamental matrix degeneration, matching ambiguities caused by repeated patterns and rejection of initially mismatched features without further reconsideration. In this paper we introduce a novel matching pipeline, which addresses all of the aforementioned challenges at once: First, we perform iterative rematching to give mismatched feature points a further chance for being considered in later processing steps. Thereby, we are searching for inliers that exhibit the same homographic transformation per iteration. The resulting homographic decomposition is used for refining matches, occlusion detection (e.g. due to parallaxes) and extrapolation of additional features in critical image areas. Furthermore, Delaunay triangulation of the matching set is utilized to minimize the repeated pattern problem and to implement focused matching . Doing so, enables us to further increase matching quality by concentrating on local image areas, defined by the triangular mesh. We present and discuss experimental results with multiple real-world matching datasets. Our contributions, besides improving matching recall and precision for image processing applications in general, also relate to use cases in image-based computer graphics.
@article{Seibt2022DFM, author = {Seibt, Simon and Von Rymon Lipinski, Bartosz and Latoschik, Marc Erich}, journal = {IEEE Access}, title = {Dense Feature Matching Based on Homographic Decomposition}, year = {2022}, volume = {10}, pages = {21236-21249}, }
2019
-
Bartosz Rymon Lipinski, Simon Seibt, Johannes Roth, and Dominik AbéIn IEEE Conference on Games, 2019Procedural generation of virtual worlds is an important aspect of game development since decades, typically for increasing replayability or for speeding up the level creation process. However, the utilization of this potential has always been a great challenge due to the difficult controllability of the underlying algorithms or limitations to specific level geometries, like 2D regular structures. In this paper, we present a novel approach for semi-automatic generation of a wide variety of 2D/3D corridor and room systems. The underlying processing pipeline is based on a separation between a user-guided generation of a graph-based abstract level structure and a fully-automatic construction of the corresponding geometry using a pre-modeled component library. The core algorithm is built on the computation of an extended minimal spanning tree, which can be controlled by a set of intuitive vertex and edge parameters. First experimental results have shown that our incremental generation pipeline allows the efficient creation of complex indoor levels, minimizing limitations on level and game designers’ creativity.
@inproceedings{Lipinski2019LevelGraph, author = {von Rymon Lipinski, Bartosz and Seibt, Simon and Roth, Johannes and Abé, Dominik}, booktitle = {IEEE Conference on Games}, title = {Level Graph - Incremental Procedural Generation of Indoor Levels using Minimum Spanning Trees}, year = {2019}, }
2017
-
Yuan Gao, Robert Fischer, Simon Seibt, Mithil Parekh, and Jianghai LiIn Informatik 2017, 2017The increasing cyber threats require quick action from security experts to protect their industrial automation control system (IACS). For fulfilling the requirement, we propose to divided the classic cyber security analysis scope into three separated, yet interconnected domains: Threat, System and Security. Thus different groups of security professionals can work independently, and are not required to have the knowledge about the full scope. In addition, we proposed an asset-centric system architecture model to enable the modeling and simulation of attacks according to publicly known threats and vulnerabilities. Analysis based on the generated attack/defense trees can assist to manage and continuously monitor the deployed security controls. The proposed approach with tool supports reduces the workload of security experts as well as the incidents response team (IRT) towards an adaptive defense manner.
@incollection{Gao2017ISF, author = {Gao, Yuan and Fischer, Robert and Seibt, Simon and Parekh, Mithil and Li, Jianghai}, title = {Integrated Security Framework}, year = {2017}, booktitle = {Informatik 2017}, publisher = {Gesellschaft für Informatik e.V.}, pages = {961--972}, series = {LNI}, }
2016
-
Simon Seibt, Karl Waedt, Hans Delfs, and Simon OdorferIn Informatik 2016, 2016Current critical industrial infrastructure (CII) which uses industrial automation and control systems (IACS) can be a target of cyber-attacks or combined cyber-physical attacks. To ensure the security of the CII, international standards are gradually evolving. The IEC 62443-x-x series is a specialized multipart security standard for IACS. It introduces, among others, the concept of security zones and security conduits [IE09]. The security zones can be defined in a physical sense and for logical grouping. By applying the defense-in-depth concept, both definitions are relevant. The development of a realistic three-dimensional (3D) model of CII can support the subdivision for the zoning and further security analyses. The model includes the locations of security relevant physical assets, which can be grouped into security zones and linked with related security artefacts. By using a 3D model, all relevant two-dimensional views can be derived. This paper addresses the use of a 3D model to support the application of security controls and risk assessments in line with concepts elaborated by IEC 62443-4-2.
@incollection{Seibt20163DM, author = {Seibt, Simon and Waedt, Karl and Delfs, Hans and Odorfer, Simon}, title = {3D modeling of selected assets, security zones and conduits}, year = {2016}, booktitle = {Informatik 2016}, publisher = {Gesellschaft für Informatik e.V.}, pages = {571--580}, series = {LNI}, }
