Skip to content

Commit 3a38bae

Browse files
AndrewLuGitAndrewLuGit
committed
add tutorial
1 parent df9151e commit 3a38bae

File tree

4 files changed

+143
-1
lines changed

4 files changed

+143
-1
lines changed

‎docs/api/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ The C API is designed for more advanced users who would rather program functiona
4949

5050
### C Sensor Functions
5151

52+
- \subpage c-aivision "C API for AI Vision Sensors"
5253
- \subpage c-distance "C API for Distance Sensors"
5354
- \subpage c-gps "C API for GPS Sensors"
5455
- \subpage c-imu "C API for IMU Sensors"

‎docs/tutorials/topical/aivision.md

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
\page aivision AI Vision Sensor
2+
3+
\note
4+
For a full list of functions for interacting with the ADI, see its
5+
: [C API](@ref c-aivision) and [C++ API](@ref cpp-aivision).
6+
7+
## Coordinate System
8+
All objects found by the AI Vision Sensor will have x and y coordinates.
9+
The origin of its coordinate system is the top left corner of the image, with
10+
x increasing from left to right and y increasing from top to bottom. Object detected
11+
based on color descriptors, code descriptors, and AI models
12+
13+
## Color Descriptors
14+
Similar to the Vision Sensor, the AI Vision Sensor can detect objects as color blobs.
15+
Colors for blob detection can be specified to the AI Vision Sensor as color descriptors.
16+
A color descriptor consists of an RGB color, a hue range, a saturation range, and an ID.
17+
IDs can range from 1-7, hue range can range from 1-40, and saturation range can range from 0.1-1.
18+
Color objects are returned by the AI Vision Sensor with type color and the same ID as the
19+
matching color descriptor, as well as the x and y coordinates of the top-left corner of its
20+
bounding box, the width and height of the bounding box, and the angle.
21+
22+
Example Code:
23+
24+
```cpp
25+
void opcontrol() {
26+
pros::AIVision aivision(2);
27+
aivision.reset();
28+
aivision.enable_detection_types(pros::AivisionModeType::colors);
29+
pros::AIVision::Color color = {.id = 1, .red = 32, .green = 142, .blue = 194, .hue_range = 20, .saturation_range = 0.5};
30+
pros::AIVision::Color color2 = {.id = 2, .red = 236, .green = 28, .blue = 95, .hue_range = 20, .saturation_range = 0.5};
31+
pros::AIVision::Color color3 = {.id = 3, .red = 146, .green = 216, .blue = 85, .hue_range = 20, .saturation_range = 0.5};
32+
aivision.set_color(color);
33+
aivision.set_color(color2);
34+
aivision.set_color(color3);
35+
while (true) {
36+
auto objects = aivision.get_all_objects();
37+
for (auto &object : objects) {
38+
if (pros::AIVision::is_type(object, pros::AivisionDetectType::color)) {
39+
printf("color\n");
40+
printf("id %d\n", object.id);
41+
printf("%d %d %d %d %d\n", object.object.color.xoffset, object.object.color.yoffset, object.object.color.width, object.object.color.height, object.object.color.angle);
42+
}
43+
}
44+
pros::delay(20);
45+
}
46+
}
47+
```
48+
49+
## Code Descriptors
50+
The AI Vision Sensor allows blobs of different colors close to each other to be detected
51+
as a single object by using code descriptors. Code descriptors tell the AI Vision Sensor
52+
which color descriptors it should combine. Code descriptors consist of an ID, the number of
53+
color descriptors to combine, and the IDs of the color descriptors. IDs can range from 1-5,
54+
and the number of color descriptors to combine can range from 2-5.
55+
Code objects are returned by the AI Vision Sensor with type code, and are otherwise identical
56+
to color objects.
57+
58+
Example Code:
59+
60+
```cpp
61+
void opcontrol() {
62+
pros::AIVision aivision(2);
63+
aivision.reset();
64+
aivision.enable_detection_types(pros::AivisionModeType::colors);
65+
pros::AIVision::Color color = {.id = 1, .red = 32, .green = 142, .blue = 194, .hue_range = 20, .saturation_range = 0.5};
66+
pros::AIVision::Color color2 = {.id = 2, .red = 236, .green = 28, .blue = 95, .hue_range = 20, .saturation_range = 0.5};
67+
aivision.set_color(color);
68+
pros::AIVision::Code code = {.id = 1, .length = 2, .c1 = 1, .c2 = 2};
69+
aivision.set_code(code);
70+
while (true) {
71+
auto objects = aivision.get_all_objects();
72+
for (auto &object : objects) {
73+
if (pros::AIVision::is_type(object, pros::AivisionDetectType::code)) {
74+
printf("code\n");
75+
printf("id %d\n", object.id);
76+
printf("%d %d %d %d %d\n", object.object.color.xoffset, object.object.color.yoffset, object.object.color.width, object.object.color.height, object.object.color.angle);
77+
}
78+
}
79+
pros::delay(20);
80+
}
81+
}
82+
```
83+
84+
## AI Model Object Detection
85+
The AI Vision Sensor can also detect objects using an AI model, which can be more robust
86+
than simple color descriptors. There is no method for uploading your own AI model, but the
87+
model on the AI Vision Sensor can be switched between competition and classroom elements by
88+
using VEXCode V5. Objects detected using the AI model are returned by the AI Vision Sensor
89+
with type object and the ID of the object class, as well as the x and y coordinates of the top-left
90+
corner of its bounding box, the width and height of the bounding box, and the score of the detection.
91+
Score is reported as a percentage, from 0-100, with higher indicating the model has higher confidence
92+
in the detection.
93+
94+
Example Code:
95+
96+
```cpp
97+
void opcontrol() {
98+
pros::AIVision aivision(2);
99+
aivision.reset();
100+
aivision.enable_detection_types(pros::AivisionModeType::objects);
101+
while (true) {
102+
auto objects = aivision.get_all_objects();
103+
for (auto &object : objects) {
104+
if (pros::AIVision::is_type(object, pros::AivisionDetectType::object)) {
105+
printf("object\n");
106+
printf("id %d\n", object.id);
107+
printf("%d %d %d %d %d\n", object.object.element.xoffset, object.object.element.yoffset, object.object.element.width, object.object.element.height, object.object.element.score);
108+
}
109+
}
110+
pros::delay(20);
111+
}
112+
}
113+
```
114+
115+
## AprilTag Detection
116+
The AI Vision Sensor can detect AprilTags of four different families.
117+
AprilTags detected by the AI Vision Sensor are returned with type tag and the ID of the AprilTag,
118+
as well as the x and y coordinates of the four corners of the AprilTag.
119+
120+
Example Code:
121+
122+
```cpp
123+
void opcontrol() {
124+
pros::AIVision aivision(2);
125+
aivision.reset();
126+
aivision.enable_detection_types(pros::AivisionModeType::tags);
127+
aivision.set_tag_family(pros::AivisionTagFamily::tag_16H5);
128+
while (true) {
129+
auto objects = aivision.get_all_objects();
130+
for (auto &object : objects) {
131+
if (pros::AIVision::is_type(object, pros::AivisionDetectType::tag)) {
132+
printf("tag\n");
133+
printf("id %d\n", object.id);
134+
printf("%d %d %d %d %d %d %d %d\n", object.object.tag.x0, object.object.tag.y0, object.object.tag.x1, object.object.tag.y1, object.object.tag.x2, object.object.tag.y2, object.object.tag.x3, object.object.tag.y3);
135+
}
136+
}
137+
pros::delay(20);
138+
}
139+
}
140+
```

‎docs/tutorials/topical/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ If you want more information about using a particular part of the \ref api, the
33
and more detail about using the PROS API.
44

55
- \subpage adi
6+
- \subpage aivision
67
- \subpage controller
78
- \subpage display
89
- \subpage filesystem

‎pros

0 commit comments

Comments
 (0)