Difference: FRCVDataRepositoryHSVD2 (1 vs. 4)
Revision 42018-02-09 - BenBarriage
| Line: 1 to 1 | ||||||||
|---|---|---|---|---|---|---|---|---|
| ||||||||
| Line: 105 to 105 | ||||||||
| * grid13Depth.tar.gz: Grid 13 4x4 grid, depth data. * grid14Depth.tar.gz: Grid 14 4x4 grid, depth data. * grid15Depth.tar.gz: Grid 14 7x7 grid, depth data. | ||||||||
| Changed: | ||||||||
| < < | * grid16Depth.tar.gz: Grid 16 7x7 grid, depth data. | |||||||
| > > | * grid16Depth.tar.gz: Grid 16 7x7 grid, depth data. | |||||||
Revision 32017-05-31 - DamianLyons
| Line: 1 to 1 | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||
| Line: 9 to 9 | |||||||||||||||||||||
Visual homing is a bioinspired, local navigation technique used to direct a robot to a previously seen location by comparing the image of the original location with the current visual image. Prior work has shown that exploiting depth cues such as image scale or stereo-depth in homing leads to improved homing performance. In this paper, we present a stereovision database methodology for evaluating the performance of visual homing. We have collected six stereovision homing databases, three indoor and three outdoor. Results are presented for these databases to evaluate the effect of FOV on the performance of the Homing with Stereovision (HSV) algorithm. Based on over 350,000 homing trials, we show that, contrary to intuition, a panoramic field of view does not necessarily lead to the best performance and we discuss why this may be the case.
Overview of the Data Repository | |||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | In this web page we report on six stereovision visual homing database inspired by the panoramic image visual homing databases of Möller and others. Using these, we have conducted an evaluation of the effect of FOV on the performance of Nirmal & Lyons's HSV homing algorithm for a variety of visual homing missions in both indoor and outdoor situations totaling over 350,000 visual homing trials. The results indicate that while in general a wide FOV outperforms a narrower FOV, peak performance is not achieved by a panoramic FOV, and we discuss why that may be the case. This web is the data repository for the databases used in the paper. | ||||||||||||||||||||
| > > | This web page is a repository for six stereovision visual homing database inspired by the panoramic image visual homing databases of Möller and others. Using these databases, we have conducted an evaluation of the effect of FOV on the performance of Nirmal & Lyons's HSV homing algorithm for a variety of visual homing missions in both indoor and outdoor situations totaling over 350,000 visual homing trials. The results indicate that while in general a wide FOV outperforms a narrower FOV, peak performance is not achieved by a panoramic FOV, and we discuss why that may be the case. This web is the data repository for the databases used in the paper.
The six databases are briefly overviewed in Table 1 below. The picture to the left on each row is a single representative picture from the database.
 Data Collection ProcedureA spatial grid is superimposed on the area to be recorded, and panoramic visual imagery is collected at each grid cell. The information is used as follows. A robot begins a simulation run with a home position grid cell and a start position grid cell. The visual information the robot receives is the stored imagery for the grid cell it occupies. A motion vector is calculated by comparing this imagery to the home grid cell imagery. This motion may move the robot to another grid cell, and the imagery comparison and motion calculation continues. Homing algorithms different in how these comparisons are done, how the motion vector is calculated, and how termination is decided. The robot used in our visual homing research is a Pioneer 3-AT robot with a Bumblebee2 stereo-camera mounted on a Pan-Tilt (PT) base. The Bumblebee2 with 3.8 mm lens has a 66 deg horizontal FOV for each camera. This limited FOV is one key challenge in building a visual homing database since each grid cell in the database must contain 360 deg of imagery; A robot can arrive at a grid location with any orientation and the simulation should be able to retrieve the imagery for that orientation. This issue is addressed by using the PT base to collect 10 visual and stereo depth images at orientations 36 deg apart, starting at 0 deg with respect to the X axis. The visual image stored is the left image from the stereo pair. The RH coordinate frame has the X axis along the direction in which the robot is facing, the Y pointing left, and is centered on the robot. The final angle requires the robot to be reoriented (in addition to the PT unit motion) due to PT unit pan limits. | ||||||||||||||||||||
| Deleted: | |||||||||||||||||||||
| < < | The six databases are briefly described in Table 1 below. | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < |
| ||||||||||||||||||||
| > > | <-- | ||||||||||||||||||||
| Added: | |||||||||||||||||||||
| > > |  Figure 1: (a, c) Set of visual images (displayed keystone warped only for display purposes) for a single square on the G15 and G14 databases and (b, d) point cloud display of all the stereo depth information from directly overhead (the XY plane) for the same squares, respectively. | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | |||||||||||||||||||||
| > > | Each visual image is a 1024x768 8-bit gray level image I_g, and each depth image I_d is a listing of the robot-centered coordinates for each point in the 1024x768 for which stereo disparity can be calculated. If (u,v) in N^2 are pixel coordinates in the gray level image, then a non-zero I_s(u,v)=(x,y,z) are the robot-centered coordinates of the point in the scene imaged by pixel I_g(u,v). The overlap in horizontal FOV (HFOV) between adjacent images is just less than 50%. | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | Integrating Stereovision Database into the HSV program | ||||||||||||||||||||
| > > | The first step in collecting a visual homing database for a physical location is to lay out a regular, square grid in the location. Our established procedure is to lay out the grid using a measuring tape, marking each grid cell on the vertex of the grid with tape. The robot is teleoperated to the vicinity of the first grid cell and its position and orientation manually adjusted so that fiducial markings on the body of the robot line up with the tape. The automated recording and storage of the imagery for the current grid cell is then initiated. When recording is finished, the robot is teleoperated to the vicinity of the next grid cell and the procedure repeated. | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | Execution begins as the program recieves coordinates for both the current and goal locations, along with the orientation of the robot in each of these instances. Just prior to this, the user will be tasked with entering the grid number of the database which they would like to use. When entering the coordinates, make sure the second value is negative; this is due to the nature of the robot's axis. | ||||||||||||||||||||
| > > | |||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | COORDINATE FRAME CONVENTIONS: A zero pose of the robot has it at the origin facing +X direction. Thus a +ve angle is CCW and turns towards +ve Y and a -ve angle is CW and turns towards -ve Y. | ||||||||||||||||||||
| > > | 
 Figure 2: Grid of all 160 images for the G12 and G14 databases. Each grid cell shows the 10 visual images for that cell in the same format as Figure 1(a), and for each database the cells are arranged in the spatial order and orientation they were collected. | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | The orientation and location will then be used to determine the proper images to use with the database grid, along with a selection of these images to represent the forward facing one hundred eighty degrees of the robot. These images are then concatenated into two panoramic images, one representing the goal location and one representing the current location, which are examined using sift features to find keypoints which can be compared between the images of the two locations. Once these keypoints are compared and a subset are determined to be valid, motion can be determined by comparing the theta and distance values for each of the matches points. Both of these calculations are performed within vh.cpp and the averages are used to determine the motion of the robot. | ||||||||||||||||||||
| > > | The dimension n of the (square) grid is the number of vertices along one side. The resolution r of the grid is the distance between grid vertices in meters. The grid cells are numbered in row-major order: A grid cell (i, j) in {0, …, n-1}^2 corresponds to database square k=(j+(n–1)i) in {0, ..., (n-1)^2}, and the database folder SQUARE_k contains the 10 visual and 10 depth images for that grid cell. The spatial location of grid cell (i, j) is given by p=(x, y)=(ir, jr) in {0, …, (n–1)r}^2. Any position p=(x, y) can be translated to its grid coordinates (i, j)=(x div r, y div r), for 0≤ x, y ≤ (n–1)r. | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | It is important to note that the distance should be calculated using the Euclidian formula for distance and that a negative value should be reachable. The robot will move a portion of the way so that compensation can be performed in the instance of an incorrect turn or move, which will occasionally occur during execution. Attention should be paid to the scale and gain values within main.cpp which will have a significant affect on the program; generally, the scale should be 1000 and the gain values should be between 0.3-0.6, but no higher than 1.0. In addition, ample time and steps must be provided because the robot may still reach its location despite taking a great deal of time to do so. | ||||||||||||||||||||
| > > |  Figure 3: Set of all 10 visual images (displayed keystone warped only for display purposes) for a single cell from the G11 (top left) through G16 (bottom right)) databases respectively. | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | Repository Information | ||||||||||||||||||||
| > > | 
 Figure 3.4: (left) A composite point cloud created from all 160 clouds (each in the form Figure 1(b)) of G14 and registered only using the spatial relationship of the grid; (right) shows this for the 490 clouds in G16. The grid of darker circles superimposed on the data in the foreground of each shows the grid vertices | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | 1. Files below labelled GRIDXX_grids contain visual image folders, while those labelled GRIDXX_cleanDump contain stereo depth data. | ||||||||||||||||||||
| > > | |||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | 2. Histogram smoothing was performed on the visual images of GRID14, an outdoor grid, in order to compensate for lighting conditions. | ||||||||||||||||||||
| > > | A note on folder naming and orientation labeling orientation labeling | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | 3. A statistical filtering program, from the point cloud library, was executed on stereo-depth to clean up some stereo noise data. The parameters of the point cloud statistical filter were a meanK of 50 and a standard deviation threshold of 0.2. | ||||||||||||||||||||
| > > | The depth files and the visual files are stored separately. For example, the depth files for G11 are stored in a folder called GRID11 with subfolders SQUARE1 through SQUARE16. The visual data for G11 is also stored in a folder called GRID11 with subfolders SQUARE1 through SQUARE16. So to download the the data, its best to make two folders Visual and Depth (for example) and download all the visual data to one and the depth to the others -- since the visual and depth folders for each database otherwise will have the same name! | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | Project Figures & Diagrams | ||||||||||||||||||||
| > > | Each set of visual images for a cell is 7.5 MB in size and each set of depth images is 200 MB on average. | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < |
| ||||||||||||||||||||
| > > | In the Visual folder GRID11, subfolders SQUARE1 to SQUARE16, the images are labelled IMAGEnnn.pgm where nnn is 0, 36, 72, 108, 144, 180, 216, 252, 288, 324. These angles are with respect to the full positive pan angle, so 0 is actually 180 deg with respect to the X axis of the robot coordinate frame. | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < |
| ||||||||||||||||||||
| > > | In the Depth folder GRID11, subfolders SQUARE1 to SQUARE16, the images are labelled dataDump_nnn_mmm.txt and these are text files. Each line the text file has the entries: X,Y,Z are the 3D coordinates of the point with respect to the robot coordinate frame, u,v are the image coordinates for the point (and used to register this point with the visual image) d is the stereo disparity r, g, b is the pixel color. | ||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | * Side by Side Comparison of HSVD Frame & Depth Images: GRID 14  | ||||||||||||||||||||
| > > | The number is as follows: nn is 0, 36, 72, 108, 144, -144, -108, -72, -36 and where mm is 0 for all except one file which is nnn=-144 and mmm=-36 which marks the one rotation which the pan unit was at its max -144 and the robot based was rotated -36 to bring the robot to 180. The angle nnn in this case is with respect to the X axis of the robot coordinate frame. Its awkward that this is not the same labeling as for the visual data, and we will revise that in the next release. Histogram smoothing was performed on the visual images of all outdoor grids, in order to compensate for lighting conditions. A statistical filtering program, from the point cloud library, was executed on all stereo-depth files to clean up some stereo noise. The parameters of the point cloud statistical filter were a meanK of 50 and a standard deviation threshold of 0.2. | ||||||||||||||||||||
Datasets | |||||||||||||||||||||
| Changed: | |||||||||||||||||||||
| < < | * GRID13_grids.tar.gz: Grid folder for HSVD project. 10/27/16 * GRID13_cleanDump.tar.gz: Clean Data Dump folder for HSVD project. 11/7/16 * GRID12_cleanDump.tar.gz: Clean Data Dump folder for HSVD project. 11/7/16 * GRID11_cleanDump.tar.gz: Clean Data Dump folder for HSVD project. 11/7/16 * GRID14_cleanDump.tar.gz: Clean Data Dump folder for HSVD project. 11/7/16 * GRID14_grids.tar.gz: Grid folder for HSVD project post smoothing. 12/5/16 * GRID12_grids.tar.gz: Grid folder for HSVD project post smoothing. 12/5/16 * GRID11_grids.tar.gz: Grid folder for HSVD project. 2/23/17 | ||||||||||||||||||||
| > > | The visual data* grid11Image.tar.gz: Grid 11 4x4 grid image data.* grid12Image.tar.gz: Grid 12 4x4 grid, image data. * grid13Image.tar.gz: Grid 13 4x4 grid, image data. * grid14Image.tar.gz: Grid 14 4x4 grid, image data. * grid15Image.tar.gz: Grid 14 7x7 grid, image data. * grid16Image.tar.gz: Grid 16 7x7 grid, image data. The depth data* grid11Depth.tar.gz: Grid 11 4x4 grid depth data.* grid12Depth.tar.gz: Grid 12 4x4 grid, depth data. * grid13Depth.tar.gz: Grid 13 4x4 grid, depth data. * grid14Depth.tar.gz: Grid 14 4x4 grid, depth data. * grid15Depth.tar.gz: Grid 14 7x7 grid, depth data. * grid16Depth.tar.gz: Grid 16 7x7 grid, depth data. | ||||||||||||||||||||
| This data is provided for general use without any warranty or support. Please send any email questions to dlyons@fordham.edu, bbarriage@fordham.edu, and ldelsignore@fordham.edu. | |||||||||||||||||||||
| Added: | |||||||||||||||||||||
| > > | The short URL for this page is http://goo.gl/h3pU7Q | ||||||||||||||||||||
Permissions* Persons/group who can change the page:<br /> * Set ALLOWTOPICCHANGE = FRCVRoboticsGroup | |||||||||||||||||||||
| Line: 77 to 129 | |||||||||||||||||||||
| |||||||||||||||||||||
| Added: | |||||||||||||||||||||
| > > |
| ||||||||||||||||||||
Revision 22017-05-31 - DamianLyons
Revision 12017-04-25 - BenBarriage
| Line: 1 to 1 | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Added: | |||||||||
| > > |
Visual Homing with Sterevision Databases (HSVD)The HSVD project is an extension of the work previously done with HSV, a method of navigation devleoped for a robotic platform based upon common navigation techniques of insects. This extension results in an HSV program that is compatible with a user's database in order for simulated navigation of a grid between a starting location and a goal location, using their respective images.This is perfomed by comparing key features found belonging to both a concatenated set of home images and a concatenated set of current location images. Through this we hope to create a working simulation of accurate and efficient traversal of a database, consisting of location coordinates and orientations, using some of the key concepts implemented earlier in the HSV project.Integrating Stereovision Database into the HSV programExecution begins as the program recieves coordinates for both the current and goal locations, along with the orientation of the robot in each of these instances. Just prior to this, the user will be tasked with entering the grid number of the database which they would like to use. When entering the coordinates, make sure the second value is negative; this is due to the nature of the robot's axis. COORDINATE FRAME CONVENTIONS: A zero pose of the robot has it at the origin facing +X direction. Thus a +ve angle is CCW and turns towards +ve Y and a -ve angle is CW and turns towards -ve Y. The orientation and location will then be used to determine the proper images to use with the database grid, along with a selection of these images to represent the forward facing one hundred eighty degrees of the robot. These images are then concatenated into two panoramic images, one representing the goal location and one representing the current location, which are examined using sift features to find keypoints which can be compared between the images of the two locations. Once these keypoints are compared and a subset are determined to be valid, motion can be determined by comparing the theta and distance values for each of the matches points. Both of these calculations are performed within vh.cpp and the averages are used to determine the motion of the robot. It is important to note that the distance should be calculated using the Euclidian formula for distance and that a negative value should be reachable. The robot will move a portion of the way so that compensation can be performed in the instance of an incorrect turn or move, which will occasionally occur during execution. Attention should be paid to the scale and gain values within main.cpp which will have a significant affect on the program; generally, the scale should be 1000 and the gain values should be between 0.3-0.6, but no higher than 1.0. In addition, ample time and steps must be provided because the robot may still reach its location despite taking a great deal of time to do so.Repository Information1. Files below labelled GRIDXX_grids contain visual image folders, while those labelled GRIDXX_cleanDump contain stereo depth data. 2. Histogram smoothing was performed on the visual images of GRID14, an outdoor grid, in order to compensate for lighting conditions. 3. A statistical filtering program, from the point cloud library, was executed on stereo-depth to clean up some stereo noise data. The parameters of the point cloud statistical filter were a meanK of 50 and a standard deviation threshold of 0.2.Project Figures & Diagrams
Datasets* GRID13_grids.tar.gz: Grid folder for HSVD project. 10/27/16* GRID13_cleanDump.tar.gz: Clean Data Dump folder for HSVD project. 11/7/16 * GRID12_cleanDump.tar.gz: Clean Data Dump folder for HSVD project. 11/7/16 * GRID11_cleanDump.tar.gz: Clean Data Dump folder for HSVD project. 11/7/16 * GRID14_cleanDump.tar.gz: Clean Data Dump folder for HSVD project. 11/7/16 * GRID14_grids.tar.gz: Grid folder for HSVD project post smoothing. 12/5/16 * GRID12_grids.tar.gz: Grid folder for HSVD project post smoothing. 12/5/16 * GRID11_grids.tar.gz: Grid folder for HSVD project. 2/23/17 This data is provided for general use without any warranty or support. Please send any email questions to dlyons@fordham.edu, bbarriage@fordham.edu, and ldelsignore@fordham.edu. Permissions* Persons/group who can change the page:<br /> * Set ALLOWTOPICCHANGE = FRCVRoboticsGroup -- (c) Fordham University Robotics and Computer Vision
| ||||||||
View topic | History: r4 < r3 < r2 < r1 | More topic actions...
Copyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding TWiki? Send feedback