Skip to content

Update vision service Go examples for NamedImage parameter #5031

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
shannonbradshaw wants to merge 1 commit into
base: main
Choose a base branch
from
+63 −68
Open

Update vision service Go examples for NamedImage parameter #5031

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions docs/vision/classify.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,12 +188,8 @@ images, _, err := cam.Images(ctx, nil, nil)
if err != nil {
logger.Fatal(err)
}
img, err := images[0].Image(ctx)
if err != nil {
logger.Fatal(err)
}

classifications, err := classifier.Classifications(ctx, img, 5, nil)
classifications, err := classifier.Classifications(ctx, &images[0], 5, nil)
if err != nil {
logger.Fatal(err)
}
Expand Down
8 changes: 2 additions & 6 deletions docs/vision/object-detection/detect.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,18 +190,14 @@ if err != nil {
logger.Fatal(err)
}

// Capture an image first
// Capture an image from the camera
images, _, err := cam.Images(ctx, nil, nil)
if err != nil {
logger.Fatal(err)
}
img, err := images[0].Image(ctx)
if err != nil {
logger.Fatal(err)
}

// Run detection on the captured image
detections, err := detector.Detections(ctx, img, nil)
detections, err := detector.Detections(ctx, &images[0], nil)
if err != nil {
logger.Fatal(err)
}
Expand Down
117 changes: 60 additions & 57 deletions static/include/services/apis/generated/vision.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ Get a list of detections from the next image from a specified camera using a con

**Returns:**

- ([List[viam.proto.service.vision.Detection]](https://python.viam.dev/autoapi/viam/proto/service/vision/index.html#viam.proto.service.vision.Detection)): : A list of 2D bounding boxes, their labels, and the
confidence score of the labels, around the found objects in the next 2D image
from the given camera, with the given detector applied to it.
- ([List[viam.proto.service.vision.Detection]](https://python.viam.dev/autoapi/viam/proto/service/vision/index.html#viam.proto.service.vision.Detection)): : A list of 2D bounding boxes, their labels, and the
confidence score of the labels, around the found objects in the next 2D image
from the given camera, with the given detector applied to it.

**Raises:**

Expand Down Expand Up @@ -78,13 +78,13 @@ For more information, see the [Go SDK Docs](https://pkg.go.dev/go.viam.com/rdk/s

**Returns:**

- (Promise<[visionApi](https://ts.viam.dev/modules/visionApi.html).[Detection](https://ts.viam.dev/classes/visionApi.Detection.html)[]>): * The list of Detections.
- (Promise<[visionApi](https://ts.viam.dev/modules/visionApi.html).[Detection](https://ts.viam.dev/classes/visionApi.Detection.html)[]>): \* The list of Detections.

**Example:**

```ts {class="line-numbers linkable-line-numbers"}
const vision = new VIAM.VisionClient(machine, 'my_vision');
const detections = await vision.getDetectionsFromCamera('my_camera');
const vision = new VIAM.VisionClient(machine, "my_vision");
const detections = await vision.getDetectionsFromCamera("my_camera");
```

For more information, see the [TypeScript SDK Docs](https://ts.viam.dev/classes/VisionClient.html#getdetectionsfromcamera).
Expand Down Expand Up @@ -128,9 +128,9 @@ Get a list of detections from a given image using a configured [detector](/refer

**Returns:**

- ([List[viam.proto.service.vision.Detection]](https://python.viam.dev/autoapi/viam/proto/service/vision/index.html#viam.proto.service.vision.Detection)): : A list of 2D bounding boxes, their labels, and the
confidence score of the labels, around the found objects in the next 2D image
from the given camera, with the given detector applied to it.
- ([List[viam.proto.service.vision.Detection]](https://python.viam.dev/autoapi/viam/proto/service/vision/index.html#viam.proto.service.vision.Detection)): : A list of 2D bounding boxes, their labels, and the
confidence score of the labels, around the found objects in the next 2D image
from the given camera, with the given detector applied to it.

**Raises:**

Expand Down Expand Up @@ -158,7 +158,7 @@ For more information, see the [Python SDK Docs](https://python.viam.dev/autoapi/
**Parameters:**

- `ctx` [(Context)](https://pkg.go.dev/context#Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
- `img` [(image.Image)](https://pkg.go.dev/image#Image): The image in which to look for detections.
- `img` [(\*camera.NamedImage)](https://pkg.go.dev/go.viam.com/rdk/components/camera#NamedImage): The image in which to look for detections.
- `extra` [(map[string]interface{})](https://go.dev/blog/maps): Extra options to pass to the underlying RPC call.

**Returns:**
Expand All @@ -169,23 +169,25 @@ For more information, see the [Python SDK Docs](https://python.viam.dev/autoapi/
**Example:**

```go {class="line-numbers linkable-line-numbers"}
// add "go.viam.com/rdk/utils" to imports to use this code snippet

myCam, err := camera.FromProvider(machine, "my_camera")
if err != nil {
logger.Error(err)
return
}
// Get an image from the camera decoded as an image.Image
img, err = camera.DecodeImageFromCamera(context.Background(), utils.MimeTypeJPEG, nil, myCam)
// Capture an image from the camera
images, _, err := myCam.Images(context.Background(), nil, nil)
if err != nil {
logger.Error(err)
return
}

myDetectorService, err := vision.FromProvider(machine, "my_detector")
if err != nil {
logger.Error(err)
return
}
// Get the detections from the image
detections, err := myDetectorService.Detections(context.Background(), img, nil)
detections, err := myDetectorService.Detections(context.Background(), &images[0], nil)
if err != nil {
logger.Fatalf("Could not get detections: %v", err)
}
Expand All @@ -210,20 +212,20 @@ For more information, see the [Go SDK Docs](https://pkg.go.dev/go.viam.com/rdk/s

**Returns:**

- (Promise<[visionApi](https://ts.viam.dev/modules/visionApi.html).[Detection](https://ts.viam.dev/classes/visionApi.Detection.html)[]>): * The list of Detections.
- (Promise<[visionApi](https://ts.viam.dev/modules/visionApi.html).[Detection](https://ts.viam.dev/classes/visionApi.Detection.html)[]>): \* The list of Detections.

**Example:**

```ts {class="line-numbers linkable-line-numbers"}
const camera = new VIAM.CameraClient(machine, 'my_camera');
const vision = new VIAM.VisionClient(machine, 'my_vision');
const camera = new VIAM.CameraClient(machine, "my_camera");
const vision = new VIAM.VisionClient(machine, "my_vision");

const { images } = await camera.getImages();
const detections = await vision.getDetections(
images[0].image,
600,
600,
images[0].mimeType
images[0].mimeType,
);
```

Expand Down Expand Up @@ -270,7 +272,7 @@ Get a list of classifications from the next image from a specified camera using

**Returns:**

- ([List[viam.proto.service.vision.Classification]](https://python.viam.dev/autoapi/viam/proto/service/vision/index.html#viam.proto.service.vision.Classification)): : The list of Classifications.
- ([List[viam.proto.service.vision.Classification]](https://python.viam.dev/autoapi/viam/proto/service/vision/index.html#viam.proto.service.vision.Classification)): : The list of Classifications.

**Example:**

Expand Down Expand Up @@ -331,15 +333,15 @@ For more information, see the [Go SDK Docs](https://pkg.go.dev/go.viam.com/rdk/s

**Returns:**

- (Promise<[visionApi](https://ts.viam.dev/modules/visionApi.html).[Classification](https://ts.viam.dev/classes/visionApi.Classification.html)[]>): * The list of Classifications.
- (Promise<[visionApi](https://ts.viam.dev/modules/visionApi.html).[Classification](https://ts.viam.dev/classes/visionApi.Classification.html)[]>): \* The list of Classifications.

**Example:**

```ts {class="line-numbers linkable-line-numbers"}
const vision = new VIAM.VisionClient(machine, 'my_vision');
const vision = new VIAM.VisionClient(machine, "my_vision");
const classifications = await vision.getClassificationsFromCamera(
'my_camera',
10
"my_camera",
10,
);
```

Expand Down Expand Up @@ -386,7 +388,7 @@ Get a list of classifications from a given image using a configured [classifier]

**Returns:**

- ([List[viam.proto.service.vision.Classification]](https://python.viam.dev/autoapi/viam/proto/service/vision/index.html#viam.proto.service.vision.Classification)): : The list of Classifications.
- ([List[viam.proto.service.vision.Classification]](https://python.viam.dev/autoapi/viam/proto/service/vision/index.html#viam.proto.service.vision.Classification)): : The list of Classifications.

**Example:**

Expand All @@ -410,7 +412,7 @@ For more information, see the [Python SDK Docs](https://python.viam.dev/autoapi/
**Parameters:**

- `ctx` [(Context)](https://pkg.go.dev/context#Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
- `img` [(image.Image)](https://pkg.go.dev/image#Image): The image in which to look for classifications.
- `img` [(\*camera.NamedImage)](https://pkg.go.dev/go.viam.com/rdk/components/camera#NamedImage): The image in which to look for classifications.
- `n` [(int)](https://pkg.go.dev/builtin#int): The number of classifications to return. For example, if you specify `3` you will get the top three classifications with the greatest confidence scores.
- `extra` [(map[string]interface{})](https://go.dev/blog/maps): Extra options to pass to the underlying RPC call.

Expand All @@ -422,23 +424,25 @@ For more information, see the [Python SDK Docs](https://python.viam.dev/autoapi/
**Example:**

```go {class="line-numbers linkable-line-numbers"}
// add "go.viam.com/rdk/utils" to imports to use this code snippet

myCam, err := camera.FromProvider(machine, "my_camera")
if err != nil {
logger.Error(err)
return
}
// Get an image from the camera decoded as an image.Image
img, err = camera.DecodeImageFromCamera(context.Background(), utils.MimeTypeJPEG, nil, myCam)
// Capture an image from the camera
images, _, err := myCam.Images(context.Background(), nil, nil)
if err != nil {
logger.Error(err)
return
}

myClassifierService, err := vision.FromProvider(machine, "my_classifier")
if err != nil {
logger.Error(err)
return
}
// Get the 2 classifications with the highest confidence scores from the image
classifications, err := myClassifierService.Classifications(context.Background(), img, 2, nil)
classifications, err := myClassifierService.Classifications(context.Background(), &images[0], 2, nil)
if err != nil {
logger.Fatalf("Could not get classifications: %v", err)
}
Expand All @@ -464,21 +468,21 @@ For more information, see the [Go SDK Docs](https://pkg.go.dev/go.viam.com/rdk/s

**Returns:**

- (Promise<[visionApi](https://ts.viam.dev/modules/visionApi.html).[Classification](https://ts.viam.dev/classes/visionApi.Classification.html)[]>): * The list of Classifications.
- (Promise<[visionApi](https://ts.viam.dev/modules/visionApi.html).[Classification](https://ts.viam.dev/classes/visionApi.Classification.html)[]>): \* The list of Classifications.

**Example:**

```ts {class="line-numbers linkable-line-numbers"}
const camera = new VIAM.CameraClient(machine, 'my_camera');
const vision = new VIAM.VisionClient(machine, 'my_vision');
const camera = new VIAM.CameraClient(machine, "my_camera");
const vision = new VIAM.VisionClient(machine, "my_vision");

const { images } = await camera.getImages();
const classifications = await vision.getClassifications(
images[0].image,
600,
600,
images[0].mimeType,
10
10,
);
```

Expand Down Expand Up @@ -525,7 +529,7 @@ Get a list of 3D point cloud objects and associated metadata in the latest pictu

**Returns:**

- ([List[viam.proto.common.PointCloudObject]](https://python.viam.dev/autoapi/viam/proto/common/index.html#viam.proto.common.PointCloudObject)): : The pointcloud objects with metadata.
- ([List[viam.proto.common.PointCloudObject]](https://python.viam.dev/autoapi/viam/proto/common/index.html#viam.proto.common.PointCloudObject)): : The pointcloud objects with metadata.

**Example:**

Expand Down Expand Up @@ -556,7 +560,7 @@ For more information, see the [Python SDK Docs](https://python.viam.dev/autoapi/

**Returns:**

- [([]*viz.Object)](https://pkg.go.dev/go.viam.com/rdk/vision#Object): A list of point clouds and associated metadata like the center coordinates of each point cloud.
- [([]\*viz.Object)](https://pkg.go.dev/go.viam.com/rdk/vision#Object): A list of point clouds and associated metadata like the center coordinates of each point cloud.
- [(error)](https://pkg.go.dev/builtin#error): An error, if one occurred.

**Example:**
Expand Down Expand Up @@ -590,14 +594,13 @@ For more information, see the [Go SDK Docs](https://pkg.go.dev/go.viam.com/rdk/s

**Returns:**

- (Promise<[commonApi](https://ts.viam.dev/modules/commonApi.html).[PointCloudObject](https://ts.viam.dev/classes/commonApi.PointCloudObject.html)[]>): * The list of PointCloudObjects.
- (Promise<[commonApi](https://ts.viam.dev/modules/commonApi.html).[PointCloudObject](https://ts.viam.dev/classes/commonApi.PointCloudObject.html)[]>): \* The list of PointCloudObjects.

**Example:**

```ts {class="line-numbers linkable-line-numbers"}
const vision = new VIAM.VisionClient(machine, 'my_vision');
const pointCloudObjects =
await vision.getObjectPointClouds('my_camera');
const vision = new VIAM.VisionClient(machine, "my_vision");
const pointCloudObjects = await vision.getObjectPointClouds("my_camera");
```

For more information, see the [TypeScript SDK Docs](https://ts.viam.dev/classes/VisionClient.html#getobjectpointclouds).
Expand Down Expand Up @@ -646,9 +649,9 @@ Used for visualization.

**Returns:**

- ([viam.services.vision.vision.CaptureAllResult](https://python.viam.dev/autoapi/viam/services/vision/vision/index.html#viam.services.vision.vision.CaptureAllResult)): : A class that stores all potential returns from the vision service.
It can return the image from the camera along with its associated detections, classifications,
and objects, as well as any extra info the model may provide.
- ([viam.services.vision.vision.CaptureAllResult](https://python.viam.dev/autoapi/viam/services/vision/vision/index.html#viam.services.vision.vision.CaptureAllResult)): : A class that stores all potential returns from the vision service.
It can return the image from the camera along with its associated detections, classifications,
and objects, as well as any extra info the model may provide.

**Example:**

Expand Down Expand Up @@ -716,14 +719,14 @@ For more information, see the [Go SDK Docs](https://pkg.go.dev/go.viam.com/rdk/s

**Returns:**

- (Promise< { classifications: [visionApi](https://ts.viam.dev/modules/visionApi.html).[Classification](https://ts.viam.dev/classes/visionApi.Classification.html)[]; detections: [visionApi](https://ts.viam.dev/modules/visionApi.html).[Detection](https://ts.viam.dev/classes/visionApi.Detection.html)[]; extra: undefined | [Struct](https://ts.viam.dev/classes/Struct.html); image: undefined | [Image](https://ts.viam.dev/classes/cameraApi.Image.html); objectPointClouds: [commonApi](https://ts.viam.dev/modules/commonApi.html).[PointCloudObject](https://ts.viam.dev/classes/commonApi.PointCloudObject.html)[]; }, >): * The requested image, classifications, detections, and 3d point
- (Promise< { classifications: [visionApi](https://ts.viam.dev/modules/visionApi.html).[Classification](https://ts.viam.dev/classes/visionApi.Classification.html)[]; detections: [visionApi](https://ts.viam.dev/modules/visionApi.html).[Detection](https://ts.viam.dev/classes/visionApi.Detection.html)[]; extra: undefined | [Struct](https://ts.viam.dev/classes/Struct.html); image: undefined | [Image](https://ts.viam.dev/classes/cameraApi.Image.html); objectPointClouds: [commonApi](https://ts.viam.dev/modules/commonApi.html).[PointCloudObject](https://ts.viam.dev/classes/commonApi.PointCloudObject.html)[]; }, >): \* The requested image, classifications, detections, and 3d point
cloud objects.

**Example:**

```ts {class="line-numbers linkable-line-numbers"}
const vision = new VIAM.VisionClient(machine, 'my_vision');
const captureAll = await vision.captureAllFromCamera('my_camera', {
const vision = new VIAM.VisionClient(machine, "my_vision");
const captureAll = await vision.captureAllFromCamera("my_camera", {
returnImage: true,
returnClassifications: true,
returnDetections: true,
Expand Down Expand Up @@ -776,7 +779,7 @@ If you are implementing your own vision service and want to add features that ha

**Returns:**

- (Mapping[[str](https://docs.python.org/3/library/stdtypes.html#text-sequence-type-str), viam.utils.ValueTypes]): : Result of the executed command.
- (Mapping[[str](https://docs.python.org/3/library/stdtypes.html#text-sequence-type-str), viam.utils.ValueTypes]): : Result of the executed command.

**Example:**

Expand Down Expand Up @@ -835,14 +838,14 @@ For more information, see the [Go SDK Docs](https://pkg.go.dev/go.viam.com/rdk/r
```ts {class="line-numbers linkable-line-numbers"}
// Plain object (recommended)
const result = await resource.doCommand({
myCommand: { key: 'value' },
myCommand: { key: "value" },
});

// Struct (still supported)
import { Struct } from '@viamrobotics/sdk';
import { Struct } from "@viamrobotics/sdk";

const result = await resource.doCommand(
Struct.fromJson({ myCommand: { key: 'value' } })
Struct.fromJson({ myCommand: { key: "value" } }),
);
```

Expand Down Expand Up @@ -885,7 +888,7 @@ Get the `ResourceName` for this instance of the vision service.

**Returns:**

- ([viam.proto.common.ResourceName](https://python.viam.dev/autoapi/viam/proto/common/index.html#viam.proto.common.ResourceName)): : The ResourceName of this Resource.
- ([viam.proto.common.ResourceName](https://python.viam.dev/autoapi/viam/proto/common/index.html#viam.proto.common.ResourceName)): : The ResourceName of this Resource.

**Example:**

Expand Down Expand Up @@ -930,7 +933,7 @@ For more information, see the [Go SDK Docs](https://pkg.go.dev/go.viam.com/rdk/r
**Example:**

```ts {class="line-numbers linkable-line-numbers"}
vision.name
vision.name;
```

For more information, see the [TypeScript SDK Docs](https://ts.viam.dev/classes/VisionClient.html#name).
Expand Down Expand Up @@ -971,7 +974,7 @@ Fetch information about which vision methods a given vision service supports.

**Returns:**

- ([Vision.Properties](https://python.viam.dev/autoapi/viam/components/audio_input/audio_input/index.html#viam.components.audio_input.audio_input.AudioInput.Properties)): : The properties of the vision service.
- ([Vision.Properties](https://python.viam.dev/autoapi/viam/components/audio_input/audio_input/index.html#viam.components.audio_input.audio_input.AudioInput.Properties)): : The properties of the vision service.

**Example:**

Expand All @@ -994,7 +997,7 @@ For more information, see the [Python SDK Docs](https://python.viam.dev/autoapi/

**Returns:**

- [(*Properties)](https://pkg.go.dev/go.viam.com/rdk/services/vision#Properties)
- [(\*Properties)](https://pkg.go.dev/go.viam.com/rdk/services/vision#Properties)
- [(error)](https://pkg.go.dev/builtin#error): An error, if one occurred.

For more information, see the [Go SDK Docs](https://pkg.go.dev/go.viam.com/rdk/services/vision#Service).
Expand All @@ -1009,12 +1012,12 @@ For more information, see the [Go SDK Docs](https://pkg.go.dev/go.viam.com/rdk/s

**Returns:**

- (Promise< { classificationsSupported: boolean; detectionsSupported: boolean; objectPointCloudsSupported: boolean; }, >): * The properties of the vision service.
- (Promise< { classificationsSupported: boolean; detectionsSupported: boolean; objectPointCloudsSupported: boolean; }, >): \* The properties of the vision service.

**Example:**

```ts {class="line-numbers linkable-line-numbers"}
const vision = new VIAM.VisionClient(machine, 'my_vision');
const vision = new VIAM.VisionClient(machine, "my_vision");
const properties = await vision.getProperties();
```

Expand Down
Loading