本文介绍如何更好地准备在 altizure.cn 上的 3D 重建的输入数据文件。

1. 谁应该阅读本文档?

  1. 运行具有超过 20,000 个图像的项目的专业用户。
  2. 倾斜摄像机和专业制图无人机的提供商。通过遵守这里描述的所需格式,您的设备的数据可以直接供给 Altizure 处理,并且 Altizure 可以加快了您的项目的处理效率。

2. 必需输入图像

一组没有重复文件名的图像。

注意:

  1. 文件名的扩展名不区分大小写。因此,DSC0000.jpgDSC0000.JPG 被认为是 Altizure 上的同一个文件。
  2. 至少3个有效的图像文件。
  3. 支持 JPG,PNG 和 TIFF,但强烈建议使用 exif 中记录的相机信息的 JPG文件。

3. 可选输入文件

  • pose.txt 是相机外部参数的文件。此文件仅对大规模数据集有帮助和必要。姿势信息仅用于使初始数据分区加速。
  • camera.txt 是相机内在参数的文件。
  • group.txt 是摄像机分组的文件。
  • mask.txt 是蒙版图像列表文件。
  • 一组 jpg 或者 png 个是的蒙版图像,用来指定相关图像中那些像素参与三维重建,那些像素不参与重建。

注意:如果可能,我们强烈建议这些外部参数和内部参数直接写入每个图像的 exif 标签。

3.1 相机外部参数的 pose.txt 格式

文件名:pose.txt

coordinatesystem <wgs84|local|wgs84_rtk|local_rtk>
<image name> <GPS> [<Orientation>]
...

<GPS> = <Latitude> <Longitude> <Altitude> 或者 <GPS> = <easting> <northing> <height>

<Orientation> = <Roll> <Pitch> <Yaw>

第一行的 coordinatesystem local 表示输入的相机位置的坐标系是用户自己定义的,不是GPS坐标系(WGS84)。如果要使用GPS坐标系(WGS84),可以直接省略第一行。第一行还可以定义坐标系为 local_rtk, wgs84_rtk,分别代表局部坐标系或者wgs84坐标系的 rtk 测量的高精度定位数据,使用这类数据,可以激活 Altizure 三维重建引擎中的 RTK 优化功能,提高无相控成果的绝对精度。

<image name>是图像的文件名,同时保证所有图像的文件名都__不重名__,如果文件名中有空格,需要用英文的双引号引住,例如"A IMAGENAME"
  <GPS>表示相机的纬度、经度、海拔坐标,在此文件中是必须申明的。如果 coordinatesystemlocal 或者 local_rtk,请按 <东距> <北距> <高程> 填写坐标信息。
<Orientation>表示相机的朝向,以度为单位计量,在此文件中是可以省略的。

例如,以下三个文件有效:

A0001.JPG 23.160948060 113.4292872428 161.6660766602 95.63634782 23.47147433 17.40054218
A0001.JPG 23.160948060 113.4292872428 161.6660766602
coordinatesystem local
A0001.JPG 1.0 1.0 1.0

A0001.JPG是图像文件名,23.160948060 113.4292872428 161.6660766602是GPS的纬度、经度、海拔坐标,95.63634782 23.47147433 17.40054218是相机姿态的旋转角度。第三个文件使用用户定义的局部坐标系,也是有效的。

以下两个文件无效:

A0001.JPG 0.9563634782 0.2347147433 0.1740054218
A0001.JPG 23.1609480602 113.4292872428 161.6660766602
A0001.JPG 23.1604395330 113.4293176755 161.5956573486

第一个文件只有相机朝向而没有GPS坐标。第二个文件有两个图像文件名重名(A0001.JPG)。

有关相机朝向的定义,请参阅 https://sidvind.com/wiki/Yaw,_pitch,_roll_camera

3.2 相机内在参数的 camera.txt 格式

文件名:camera.txt

<image name> <focal length> 或者 <image name> <focal length> <image center x> <image center y>

<image name>是直接图像名称,同时保证图像的所有文件名都__不重名__。
<focal length>是相机的焦距,以像素为单位计量。

如果图片所含的焦距信息是毫米焦距,这时需要转换成像素焦距来计算,转换公式如下:

  • f(像素)= w(像素) * F(毫米焦距) / w(mm)

w(像素)为照片w方向的像素
F(毫米焦距)为该镜头的焦距,单位为毫米。
w(毫米)为相机传感器(CMOS)对应长度,单位为毫米。

具有分辨率 6000×4000 35mm cmos 23.5*15.6 的图像的示例相机内在文件如下所示:

计算过程:f1=6000*25/23.5=6382.97

A0001.JPG 6382.97
A0002.JPG 6382.97 

<image center x> <image center y> 是相片的投影中心。单位是像素,坐标原点在相片的左上角,x 轴向右,y 轴向下。如果您的数据采集方或者相机厂家没有提供专门投影中心的校准参数,请不要填写这个部分。

3.3 相机分组的 group.txt 格式

文件名:group.txt

<image name> <group ID>

<group ID>是图像的分组编号,以阿拉伯数字表示(>=0)。位于同一分组的图像表示改组图像是由同一相机拍摄的。同一型号的不同相机拍摄的图片应被划分为不同分组。
这里的示例显示六张图像被分组成三组:

A0001.JPG 1
A0002.JPG 1
B0001.JPG 2
B0002.JPG 2
C0001.JPG 3
C0002.JPG 3

3.4 蒙版文件 mask.txt 格式

文件名: mask.txt
<image name> <mask name with extension>

每行指定图片 <image name> 对应的蒙版图像 <mask name with extension>
<mask name with extension> 该蒙版图像名字必须准确匹配实际的蒙版文件,包含对应的文件扩展名并且是大小写敏感的。输入文件夹中的图像如果无法在 mask.txt 里查询到对应的蒙版文件名,将被当作输入图像。蒙版图像必须是 jpg 或者 png 格式。蒙版图像中亮度为 0 的像素代表输入图像中对应位置的像素不参与三维重建,反之其他亮度的像素则代表参与三维重建。我们建议蒙版图像的分辨率与对应的输入图像一致,如果不一致,相应的蒙版图像将被放大或者缩小到对应的输入图像一致的尺寸。

4. 支持的 EXIF/XMP 标签

As we recommended, if all the camera information is written into exif/xmp data of jpg images, then Altizure can detect and extract all parameters from exif.

Camera 内方位元素

  • Exif Tag
    • FocalLength
    • FocalLengthIn35mm
  • Altizure Specific XMP Tag
    • altizure:CalibratedFocalLength (double in pixels)
    • altizure:CalibratedOpticalCenterX (double in pixels)
    • altizure:CalibratedOpticalCenterY (double in pixels)
    • altizure:CalibratedRadialDistortionK1 (ref to opencv radial distortion)
    • altizure:CalibratedRadialDistortionK2
    • altizure:CalibratedRadialDistortionK3

Camera 外方位元素

  • Exif Tag
    • GPSAltitude: GPSAltitude is the absolute altitude above the sea level in meters, which is typically measured by a barometer and can be quite inaccurate in practical use.
    • GPSRelativeAltitude: GPSRelativeAltitude is the relative altitude above the point where a UAV takes off. The measurement is generally much more accurate than GPSAltitude, since the vehicles like DJI UAVs usually use different sensors like IMU and visual tracking system to give a fused estimation of it. Therefore, we use this as the altitude measurement in our pipeline. The absolute altitude of the take-off point needs to be added to the altitude of the output model to give absolute altitude measurements of our model.
    • GPSLatitude
    • GPSLatitudeRef
    • GPSLongitude
    • GPSLongitudeRef
  • DJI Specific XMP Tag:
    • drone-dji:RtkFlag
    • drone-dji:RtkStdLon
    • drone-dji:RtkStdLat
    • drone-dji:RtkStdHgt
      Remarks
      According to the documentation of DJI Phantom4 RTK, RtkFlag marks the types of RTK solutions: 16 means point positioning (of around 3cm accuracy), 34 means float solution (of 10~40 cm accuracy) and 50 means fixed solution (of around 5cm accuracy). We also found that M210 RTK V2 may have a RtkFlag=1. For now, we take the GPS values as accurate RTK measurements if RtkFlag=1, 16 or 50. RtkStdLon, RtkStdLat, RtkStdHgt are the standard deviations of the GPS measurement.
  • senseFly Specific XMP tag:
    • Camera:GPSXYAccuracy (double in meters)
    • Camera:GPSZAccuracy (double in meters)
      Remarks
      If the accuracy is high enough, e.g. smaller than 5cm, Altizure engine will consider this GPS measurement is a fixed rtk measurement.
  • Altizure Specific XMP tag:
    • altizure:SRS (string)
    • altizure:LocalX (double in meters)
    • altizure:LocalY (double in meters)
    • altizure:LocalZ (double in meters)
    • altizure:StdX (double in meters)
    • altizure:StdY (double in meters)
    • altizure:StdZ (double in meters)
    • altizure:GravityX (double in meters/second^2)
    • altizure:GravityY (double in meters/second^2)
    • altizure:GravityZ (double in meters/second^2)
    • altizure:Extrinsic (string)
      Remarks
      The LocalX, LocalY and LocalZ specify the spatial coordinates of an image in a local right-hand-side (RHS) cartesian coordinate system. SRS (spatial reference system) can use to define more specifically how such local cartesian coordinate related to the globe. StdX, StdY, and StdZ are the standard deviation of the estimation of LocalX|Y|Z, if they are provided and are small enough, e.g. less than 0.05, but greater than 0, our engine will use LocalX|Y|Z to constraint the 3D reconstruction. Otherwise, only a similarity transformation is estimated to align the up-to-scale 3D reconstruction to the local coordinate system. GravityX|Y|Z can help our engine to align the reconstructed model to the gravity direction, but it cannot help to recover the absolute scale of the reconstructed 3D model. Such a gravity direction can usually be obtained from IMU or visual inertial algorithms. Multiple measurements of GravityX|Y|Z from the xmp information of a few images will be averaged to get a dominant gravity direction. If LocalX|Y|Z and GravityX|Y|Z are provided at the same time, GravityX|Y|Z will be ignored, because LocalX|Y|Z can provide both the gravity (up) direction and the absolute scale. Extrinsic is the extrinsic matrix for each image, which is a string encoding the 4x4 matrix in a row-major fashion. For example, "1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1" is a valid Extrinsic input. Note that this string always ends with "0 0 0 1" or something similar (double floating-point numbers). Any string with less than 16 double numbers will be considered as invalid extrinsic matrix. If Extrinsic and LocalX|Y|Z are all specified and valid at the same time, Extrinsic will be taken and LocalX|Y|Z will be ignored.

Camera 元数据

Fill in this camera meta information as much as possible. They are very helpful for our engine to divide the input images into a few camera groups each of which shares the same set of camera intrinsic in the 3D reconstruction. Our engine will automatically create a new group for an image with any of these fields, Make, Model, SerialNumber, LensSerialNumber, Lens and focal length, are different. The focal length is an estimated focal length in pixels from FocalLength and FocalLengthIn35mm, from drone-dji:CalibratedFocalLength, or from altizure:CalibratedFocalLength.

  • Make
  • Model
  • Lens
  • SerialNumber
  • LensSerialNumber