Updating Camera Calibration

wiki/sensing/camera-calibration.md

@@ -1,13 +1,29 @@
 ---
-date: 2017-08-15
+date: 2024-05-01
 title: Camera Calibration
 ---
 
-It is quite common for projects to involve vision systems, but despite this most teams working with cameras ignore the basic advantages of calibrating a camera properly. Not only does a good calibration lead to better quality of data in form of images captured but also reduces overall errors in the vision the system, a properly calibrated camera helps avoid unnecessary delays due to radiometric and geometric errors.
+It is quite common for projects to involve vision systems, but despite this many teams working with cameras ignore the basic advantages of calibrating a camera properly. Not only does a good calibration lead to better quality of data in form of images captured but also reduces overall errors in the vision the system, a properly calibrated camera helps avoid unnecessary delays due to radiometric and geometric errors.
+
+## What is Camera Calibration?
+Camera Calibration is the process of estimating internal or external parameters of a camera by taking multiple measurements of a known object. Often both are necessary to get a system working well.
+
+#### Types of Camera Calibration
+- **Intrinsic** - Estimates the internal parameters of the camera including the focal length, the optical center, and distortion coefficients. This can be useful when trying to transform points between pixel space and 3D space.
+- **Extrinsic** - Estimates the external parameters of the camera i.e. the translation and rotation of the camera relative to a reference point. This can be useful when incorporating information from multiple sensors or when trying to act upon information captured by a camera.
+
+## Tools for Camera Calibration
+There are multiple methods of camera calibration depending on what you are trying to achieve. Some specific tutorials on camera calibration are listed below:
+
+- [Camera IMU Calibration](/wiki/sensing/camera-imu-calibration.md) - Using the Kalibr library to calibrate a camera and IMU
+    - Kalibr can also be used for intrinsic camera calibration as shown in [this video](https://www.youtube.com/watch?app=desktop&v=puNXsnrYWTY)
+- [Hand-Eye Calibration with easy_handeye](/wiki/sensing/easy-handeye.md) - Using the easy_handeye library to perform extrinsic camera calibration with a robotic arm
+- [Photometric Calibration](/wiki/sensing/photometric-calibration.md) - Calibrating a camera to determine the absolute value of light intensity
+
 
 ## References
-Listed below are a few good places to get started on camera calibration.
-- Quick intro. and overview: https://www.cs.umd.edu/class/fall2013/cmsc426/lectures/camera-calibration.pdf
-- A comprehensive guide: http://www.cs.rutgers.edu/~elgammal/classes/cs534/lectures/CameraCalibration-book-chapter.pdf
-- Chapter 2 in this book: http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf (I would highly recommend this book, above all)
-- Using MATLAB for online/offline calibration: https://www.mathworks.com/help/vision/camera-calibration.html
+This is only a brief overview, for more detailed information about camera calibration, check out the links below
+- [Camera Calibration](http://www.cs.rutgers.edu/~elgammal/classes/cs534/lectures/CameraCalibration-book-chapter.pdf) - Zhengyou Zhang
+- [Computer Vision: Algorithms and Applications](http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf) - Richard Szeliski (Chapter 2)
+- [Camera Calibration](https://www.mathworks.com/help/vision/camera-calibration.html) - Mathworks
+- [What is lens distortion?](https://photographylife.com/what-is-distortion) - Nasim Mansurov 

wiki/sensing/computer-vision-considerations.md

@@ -1,5 +1,5 @@
 ---
-date: 2017-08-15
+date: 2024-05-24
 title: Computer Vision for Robotics- Practical Considerations
 ---
 Things to take care of when working with vision in robotics:
@@ -19,7 +19,7 @@ Its easy to get enchanted by the promises of the camera. However, computer visio
 - If you are doing pose estimation using CV, your calibration error will affect your pose accuracy/precision.
 - When calibrating, make sure your target (chessboard) is as planar as it can get. Also make sure its edges and corners are sharp.
   - The above point is even more relevant if you are using the OpenCV software for calibration because it auto-detects corners
-  - Apart from OpenCV, another good calibration toolbox is [Caltech Calib Toolbox](http://www.vision.caltech.edu/bouguetj/calib_doc/).
+- Refer to the [Camera Calibration Entry](/wiki/sensing/camera-calibration.md) for more information
 
 ## Scale
 - Any knowledge you recover about the world using Computer Vision is only accurate up to a scale. Estimating this scale is difficult. Possible solutions include:

wiki/sensing/easy-handeye.md

@@ -0,0 +1,54 @@
+---
+date: 2024-05-01
+title: Hand-Eye Calibration with easy_handeye
+---
+
+*[Easy Handeye](https://github.com/IFL-CAMP/easy_handeye)* from the TUM Computer Aided Medical Procedures Lab is an extremely useful tool for calibrating a camera with an arm. There are two types of hand-eye calibration:
+
+- **eye-in-hand** - The camera is mounted on the arm end-effector and it is necessary to determine the static transform between the camera frame and the end-effector.
+- **eye-on-base** - The camera is mounted in the workspace of the arm and it is necessary to determine the static transform between the camera frame and the base of the arm.
+
+## Procedure
+Print a marker from [ArUco Marker Generator](https://chev.me/arucogen/), be sure to select 'Original ArUco' for the dictionary. Measure the edge length of the marker in meters and record for later. For eye-in-hand calibration, the ArUco marker should be affixed to a static object. For eye-on-base calibration, the ArUco marker should be affixed to the end-effector.
+
+Install aruco ros
+
+```
+sudo apt-get install ros-noetic-aruco-ros
+```
+
+Clone the easy_handeye repository into a new or existing ROS workspace and make the workspace.
+
+```
+cd catkin_ws/src
+git clone [email protected]:IFL-CAMP/easy_handeye.git
+cd ../ && catkin_make
+```
+
+Select one of the [example launch files](https://github.com/IFL-CAMP/easy_handeye/tree/master/docs/example_launch), move it to `easy_handeye/easy_handeye/launch`. This launch file launches the camera, robotic arm, aruco detector, and handeye calibration tool.
+
+If you cannot find a launch file that matches your camera arm configuraiton, you will have to modify an existing one.
+
+Within the launch file, replace the camera and arm launch commands with the launch commands corresponding to your system.
+
+Within the launch file set the following parameters:
+- `eye_on_hand` - True for eye-on-hand, False for eye-on-base
+- `marker_size` - The size of your ArUco marker in meters
+- `marker_id` - The ID of your ArUco marker
+- `tracking_base_frame` - The RGB frame of your camera
+- `tracking_marker_frame`- The frame of the ArUco marker detection
+- `robot_base_frame`- The frame of the base of the robotic arm
+- `robot_effector_frame` - The frame of the end effector of the robotic arm
+
+Once all parameters are changed, run the launch file.
+
+```
+roslaunch easy_handeye <YOUR_LAUNCH_FILE>
+```
+
+Move the arm in manual mode to an angle where the camera can see the aruco marker and hit capture. Repeat this 10-20 times for different arm configurations then hit compute. Try to get diverse arm positions. This will give you the translation and rotation of the camera relative to the base or end effector depending on which calibration being performed.
+
+One thing to note is that sometimes this calibration needs to be adjusted manually by hand. If possible, verify the calibration in different arm configurations and adjust the translation and rotation if necessary.
+
+## References
+- [Easy Handeye](https://github.com/IFL-CAMP/easy_handeye)