신동원 Dong-Won Shin

I am a developer who values dedication and persistence, focusing deeply on one area to achieve mastery. Although my pace may be slow, I strive to warm those around me, much like the steady and comforting glow of a fireplace. I channel my passion for expertise into creating a positive impact on others. I actively participate in various tech community activities to embody this philosophy, and I continue to do so.
While I enjoy learning and mastering new technologies, I firmly believe that focusing on products, customers, and market needs is even more critical. Understanding market demands and creating products that meet customer expectations are, in my view, far more important than simply acquiring the latest technologies.

celinachild@gmail.com

Youtube

Facebook

Github

Career

Mar. 2022 - present

Aug. 2019 - Feb. 2022

StradVision

VSLAM Algorithm Engineer

SOSLAB (Smart Optics Sensor Lab)

Research Engineer in LiDAR Application Team

Education

Mar. 2015 - Aug. 2019

GIST (Gwangju Institute of Science and Technology)

Ph.D. degree in School of Electronic Engineering and Computer Science

Mar. 2013 - Feb. 2015

GIST (Gwangju Institute of Science and Technology)

M.S. degree in School of Electronic Engineering and Computer Science

Mar. 2006 - Feb. 2013

KIT (Kumoh National Institute of Technology)

B.S. degree in Computer Engineering (Including two years of the millitary service)

Research Interest

•

Simultaneous Localization and Mapping

•

Multi View Geometry 

•

Visual Localization

•

Computer Vision and Image Processing

•

Machine Learning and Deep Learning

•

Autonomous Driving System

•

Mobile Robotics

Technical Skills & Experience

•

Programming language: C/C++, Python

•

Robotics: ROS

•

Image processing: OpenCV

•

3D geometry: Point cloud library, Open3D

•

Machine learning: Numpy, Scipy, Scikit-learn

•

Deep learning: Tensorflow, Pytorch, Keras

•

Development: Github, Docker, CMake

•

Nonlinear optimization: Ceres, g2o, symforce

Project

Deep Learning based visual localization for automotive embedded platform

2024. Jan - 2024. Dec

Semantic aware parking slot map generation

2023. Jan - 2023. Dec

Ultra-lightweight Visual SLAM framework for Auto Valet Parking

2022. Feb - 2022. Dec

Software development of SLAM mapping device using solid-state lidar sensor

Nov. 2021 - Jan. 2022

Development of robust 3D SLAM & object detection software for the harsh environment

Mar. 2020 - Feb. 2022

Development of object detection algorithm for automotive using 3D solid-state lidar sensor

Feb. 2021 - Aug. 2021

Building datasets for indoor lidar and AR/VR artificial intelligence training

Sep. 2020 - Feb. 2021

Development of a two-dimensional lidar object recognition algorithm for factory automation in smart factories

Dec. 2021 - Jan. 2022

Creating Two-Dimensional Driving Maps for Indoor Autonomous Robots

Nov. 2021

Developing Cognitive Software for Bolt Picking Robots for Factory Automation in Smart Factories

Feb. 2020 - March. 2020

Side Project

Evaluation Automation Tool for Visual SLAM

2024

Hyper Parameter Tunning Tool for Visual SLAM

2023

NeRF Art Gallery

2022

SLAMOps: SLAM as an MLOps

Nov. 2021 - on going

NaverLabs Visual Localization Challenge

April 2020 - June 2020

Translating the Visual SLAM book

Oct. 2018 - Feb. 2019

Patent

Title

등록일자

출원일자

라이다 데이터 처리 방법 (A METHOD OF PROCESSING LIDAR DATA)

Open

2023.02.27

대상체의 기하학적 특성을 반영하여 확장된 표현 범위를 가지는 인텐시티 정보를 생성하는 방법 및 그러한 방법을 수행하는 라이다 장치 (A method of generating an intensity information with extended expression range by reflecting a geometric characteristic of object and a LiDAR device that performs the method)

Open

2021.06.25

Blog

Introduction

Recently, while studying IMU preintegration, I suddenly wondered how IMU preintegration is implemented in ORB-SLAM3, so I analyzed it. The theoretical part of IMU preintegration is already well explained Dr. Lim's blog, so I think it would be good to refer to it, and I compared the IMU preintegration paper with the ORB-SLAM3 code to see how the formula in the paper is implemented in the code.

It would take a lot of time to write the IMU preintegration part in one article, so I would like to write it in the following order.

IMU sensor measurement integration

Defining the residual function

Definition of Jacobian matrix

This article will cover the code analysis of the first of them, IMU sensor measurement integration.

Code analysis

Integrating IMU Sensor Measurements

First, let's look at the part of the code that integrates the measurements between keyframe i and keyframe j by accumulating the acceleration and angular velocity measurements from the IMU sensor as they come in. During this process, the following values are computed

The overall calculation is performed in the IntegrateNewMeasurement function of the Preintegrated class. The complete function code is shown below.

[ENG] ORB-SLAM3 IMU preintegration code review (1)

Introduction

Last time, we analyzed the first of several components of IMU preintegration in ORB-SLAM3, the IMU sensor measurement integration part, by comparing the paper and the code. The measurement integration part (1) computes the relative state (rotation, translation, velocity) values between two keyframes, (2) obtains the covariance matrix through noise propagation, and (3) computes the Jacobian matrix for bias update. The values computed here are later used to perform bundle adjustment using keyframes, or to perform nonlinear optimization such as pose estimation for a single camera frame.

In this tutorial, we will analyze how the residual function of IMU factors and the definition of Jacobian matrix are implemented in the ORB-SLAM3 code to perform nonlinear optimization.

Prerequisites

Factor Graphs and MAP Esimation

A factor graph is a structure used to represent the relationships between variables in a stochastic model and to find solutions that satisfy all constraints (factors). It is used here to address the Visual-Inertial Odometry (VIO) problem and consists of the following main components

Graph node

Graph edges

The VIO problem defined in the IMU preintegration paper is represented as follows in Equation (25).

\begin{equation} \begin{aligned} p(\mathcal{X}_k \mid \mathcal{Z}_k) &\propto p(\mathcal{X}_0) p(\mathcal{Z}_k \mid \mathcal{X}_k) \\ &\overset{(a)}{=} p(\mathcal{X}_0) \prod_{(i,j) \in \mathcal{K}_k} p(\mathcal{C}_i, \mathcal{I}_{ij} \mid \mathcal{X}_k) \\ &\overset{(b)}{=} p(\mathcal{X}_0) \prod_{(i,j) \in \mathcal{K}_k} p(\mathcal{I}_{ij} \mid \mathbf{x}_i, \mathbf{x}_j) \prod_{i \in \mathcal{K}_k} \prod_{l \in \mathcal{C}_i} p(\mathbf{z}_{il} \mid \mathbf{x}_i). \end{aligned} \tag{25} \end{equation}

where the state value

\mathcal{X}_k

the state for the keyframe we are interested in obtaining (rotation

\mathbf{R}_i

, translation

\mathbf{p}_i

, velocity

\mathbf{v}_i

, bias

\mathbf{b}_i

) values, and the measurements

\mathcal{Z}_k

contain observations from the camera sensors (2D keypoint position information

\mathcal{C_i}

about 3D landmarks) and measurements from the IMU sensors (acceleration and angular velocity

\mathcal{I_{ij}}

). For more details on the notation, please refer to the section IV. Maximum a Posteriori Visual-Inertial State Estimation in the paper.

[ENG] ORB-SLAM3 IMU preintegration code review (2)

안녕하세요 SLAM KR!
SLAM에 입문하시는 슬린이 여러분들을 위해 SLAM KR 커뮤니티에서 공부한 내용들을 교재로 정리해보았습니다 :-)
SLAM이 무엇인지 , 어디에 쓰일수 있는지 흥미를 돋우는 내용으로 시작하여, SLAM에 사용하는 센서들과 SLAM을 공부하실 때 필요한 수학적 기초 지식들(선형대수, 확률통계, 비선형 최적화, 회전변환)을 정리했습니다. 크게 Frontend와 Backend로 분리하여 SLAM을 공부하는데 필요한 내용들을 수식과 코드 설명, 실습을 포함하여 상세히 설명하고자 했습니다. 마지막으로 SLAM 기술의 미래 발전 방향과 이러한 SLAM 관련 기술들을 공부할 수 있는 커뮤니티 및 관련 자료들을 정리했습니다.
아직 부족한 부분이 많아서 공개를 해야할지 고민을 많이 하였으나  SLAM을 처음 접하시거나 SLAM에 대해 좀 더 깊이 공부하고 싶으신 분들에게 자그마한 도움이 되셨으면 좋겠다는 마음에 자료를 공개합니다. 
슬린이가 슬잘알이 되는 그 날까지!
감사합니다 :-)

Loading PDF…

LINK: http://bit.ly/3Jgq1aZ

PASSWARD: slamkr

NVIDIA Jetson Nano와 함께하는 SLAM의 이해와 구현

소개

지난 시간에 우리는 ORB-SLAM3에서 IMU preintegration의 여러 구성요소 중에 첫번째로 IMU 센서 측정 값 integration 부분을 논문과 코드를 대조하여 분석해보았습니다. 측정값 integration 부분에서는 (1) 두 키프레임 사이의 상대적인 상태 (회전, 이동, 속도) 값을 계산하고, (2) 노이즈 전파를 통해 공분산 행렬을 구하고, (3) 바이어스 업데이트를 위한 자코비안 행렬을 계산합니다. 여기서 계산된 값들은 나중에 키프레임들을 이용한 Bundle adjustment를 수행할 때나, 카메라 프레임 하나에 대한 Pose estimation등 비선형 최적화를 수행할때 사용됩니다.

이번 시간에는 비선형 최적화를 수행할때 구성하는 IMU factor의 Residual 함수와 Jacobian 행렬의 정의에 대해서 ORB-SLAM3 코드에서는 어떻게 구현되어 있는지 분석해보도록 하겠습니다.

사전 지식

Factor Graphs and MAP Esimation

Factor graph는 확률 모델에서 변수 간의 관계를 표현하고 모든 제약조건(factor)을 만족시키는 해를 찾는데 사용되는 구조입니다. 여기서는 Visual-Inertial Odometry(VIO) 문제를 다루기 위해 사용되며 아래의 주요 구성 요소로 이루어져 있습니다.

그래프 노드(Graph node)

그래프 엣지(Graph edge)

IMU preintegration 논문에서 정의한 VIO 문제는 수식 (25)에서 아래와 같이 표현됩니다.

\begin{equation} \begin{aligned} p(\mathcal{X}_k \mid \mathcal{Z}_k) &\propto p(\mathcal{X}_0) p(\mathcal{Z}_k \mid \mathcal{X}_k) \\ &\overset{(a)}{=} p(\mathcal{X}_0) \prod_{(i,j) \in \mathcal{K}_k} p(\mathcal{C}_i, \mathcal{I}_{ij} \mid \mathcal{X}_k) \\ &\overset{(b)}{=} p(\mathcal{X}_0) \prod_{(i,j) \in \mathcal{K}_k} p(\mathcal{I}_{ij} \mid \mathbf{x}_i, \mathbf{x}_j) \prod_{i \in \mathcal{K}_k} \prod_{l \in \mathcal{C}_i} p(\mathbf{z}_{il} \mid \mathbf{x}_i). \end{aligned} \tag{25} \end{equation}

여기서 상태값

\mathcal{X}_k

는 우리가 구하고자 하는 키프레임에 대한 상태(회전

\mathbf{R}_i

, 이동

\mathbf{p}_i

, 속도

\mathbf{v}_i

, 바이어스

\mathbf{b}_i

)값을 포함하고 측정값

\mathcal{Z}_k

는 카메라 센서로부터의 관찰(3D 랜드마크에 대한 2D 키포인트 위치정보

\mathcal{C_i}

)값 및 IMU 센서로부터의 측정(가속도 및 각속도

\mathcal{I_{ij}}

)값을 포함합니다. Notation에 대해서 더 자세히 알고싶으신 분들은 논문에서 IV. Maximum a Posteriori Visual-Inertial State Estimation 섹션을 참고해주세요.

[KOR] ORB-SLAM3 IMU preintegration code review (2)

소개

최근 IMU preintegration에 대해서 공부하다가 문득 ORB-SLAM3는 어떻게 IMU preintegration이 구현되어 있을지 궁금해서 분석해보았습니다. IMU preintegration에 대한 이론적인 부분은 형태님의 블로그에 이미 잘 설명이 되어있기 때문에 그 쪽을 참고하시면 좋을것 같고 저는 IMU preintegration 논문과 ORB-SLAM3 코드를 대조해보면서 논문의 수식이 코드에서 어떻게 구현되어 있는지를 살펴보았습니다.