Skip to content

Commit

Permalink
add image
Browse files Browse the repository at this point in the history
  • Loading branch information
brunosan committed Dec 26, 2023
1 parent ea1491c commit 35d0e35
Show file tree
Hide file tree
Showing 2 changed files with 64 additions and 159 deletions.
110 changes: 31 additions & 79 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
# Clay model documentation
# Clay documentation

<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

This site cover the documenation on how the Clay model is done, and how
to use it.

## Overview

Clay is a [foundational model](#) of Earth using Earth Observation data.
Expand All @@ -23,88 +20,43 @@ This fine-tuning supports various tasks, including classification
as creating RGB imagery from SAR data. Moreover, users can further
enhance model performance by incorporating higher-resolution data.

Currently in `v0`, it uses ten bands from Sentinel-2, two from
Sentinel-1, and a Digital Elevation Models (DEM).

The model code lives on
[Github](https://github.com/Clay-foundation/model), the model weights on
[Huggin Face](#), and we maintain a set of embeddgins on [Source
Cooperative](#).

Documentation for this model is made using [nbdev](), making every page
a python notebook anyone can use, practical code examples for each
functionality, and use case.
This documentation uses [nbdev](#), which combines documentation, code
samples and an SDK. This means that every page is also a python notebook
anyone can use, with practical code examples for each functionality, and
use case. Moreover, you can install `pip install clay` and use the same
functions.

Clay is open source, open data and open for business.

Clay is a fiscally sponsored project of [Radiant Earth](), a USA
registered 501(c)3 non-profit.

## How Clay is developed
## Where is what

Development of Clay model code happens in the open on [this
repo](https://github.com/Clay-foundation/model). It is licensed as
Apache.

We have high-level goals to define each version: \* `v0`: Released
Dec’23. Development version aiming to have a functional architecture,
small sample of trainning data, and notional embeddgins. See [v0 Release
notes](#). \* `v1`: Released planned for MArch ’24
- The Clay model **code** lives on
[Github](https://github.com/Clay-foundation/model). License:
[Apache](https://github.com/Clay-foundation/model/LICENSE).
- The Clay model **weights** live on [Huggin Face](#). License:
[OpenRAIL-M](https://github.com/Clay-foundation/model/blob/main/LICENSE-MODEL.md).
- The Clay **documentation** lives on here. License: [CC-BY](#).
- The Clay **SDK** lives on [PyPi](#). License: [Apache](#).
- We maintain a set of **embeddings** on [Source Cooperative](#).
License: [ODC-BY](#).

## How to use Clay

The model can be used in two main ways: 1. Directly, generating semantic
**embeddigns**. E.g. Similarity search. 3. **Fine-tunning** output of
the model: With additional data as labels, and optionally different
input data (e.g. higher-res images).: 1. Classification: E.g. flood
detection, crop types, damage assessment, deforestation monitor… 2.
Regression tasks: E.g. Carbon stock above ground, crop yields, … 3.
**Generative** tasks: E.g. Estimate RGB imagery wihtout clouds or from
Synthetic Aperture Radar (SAR) data. to leverage clay’s pre-processing
for your mode.

### Embeddings

Clay can be directly used to generate semantic embeddings. These
embeddings are powerful in understanding and interpreting complex
spatial and temporal patterns in EO data.

A prime example is “Similarity Search”: Utilize embeddings to find
similar patterns or features within large datasets. This is particularly
useful in environmental monitoring and land use classification.

See [Embeddings]()
The model can be used in two main ways:

### Fine-tunning
1. Directly, use it to make inference. See [Model](Model.html)
1. Check and run Benchmarks on the model. See
[Benchmarks](Benchmarks.html)
2. Generating semantic **embeddings**. E.g. for Similarity search. See
[Embeddings](Embeddings.html).
3. **Fine-tunning** the model for other tasks, or for other input data.
E.g. flood detection, crop yields, … See
[Fine-tunning](Fine-tunning.html).
4. **Generative** tasks: E.g. Estimate RGB imagery wihtout clouds or
from Synthetic Aperture Radar (SAR) data. See
[Generative](Generative.html).

As a foundational model, Clay is highly adaptable through fine-tuning,
allowing it to tackle specific tasks by using additional labeled data
and, optionally, different input data like higher-resolution images.

#### Classification tasks

Apply Clay to classify various geographical features and phenomena, such
as flood detection, crop type identification, damage assessment, and
deforestation monitoring.

Fine-tune Clay with labeled data relevant to your classification task.
This involves retraining the model’s final layers to recognize and
categorize specific features or conditions in EO data.

#### Regresion tasks

Estimate quantitative information like carbon stock above ground, crop
yields, and other environmental metrics.

Regression tasks require fine-tuning with data that provide a measurable
output. Clay learns to predict these continuous values from EO data,
enabling precise environmental assessments.

### Generative AI

Clay can generate new data representations, such as estimating RGB
imagery without clouds or converting SAR data into RGB imagery.

This involves training Clay to recognize patterns in SAR data and
translate them into corresponding RGB formats, a process that can be
critical in cloud-penetrated regions or for night-time imagery analysis.
Clay is a fiscally sponsored project of [Radiant Earth](), a USA
registered 501(c)3 non-profit.
113 changes: 33 additions & 80 deletions docs/nbs/Clay Model releases/Clay v0 release.ipynb

Large diffs are not rendered by default.

0 comments on commit 35d0e35

Please sign in to comment.