-
Notifications
You must be signed in to change notification settings - Fork 25
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
7 changed files
with
685 additions
and
37 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,50 +1,171 @@ | ||
# Decentraland SDK 7 | ||
# Decentraland SDK Toolchain | ||
|
||
[![codecov](https://codecov.io/gh/decentraland/js-sdk-toolchain/branch/main/graph/badge.svg?token=F7J331CGP6)](https://codecov.io/gh/decentraland/js-sdk-toolchain) | ||
This monorepo contains the core packages for Decentraland's SDK and development tools. | ||
|
||
Use the Decentraland Software Development Kit v7 to create experiences for the Decentraland ecosystem. | ||
## Packages | ||
|
||
## Create a scene and preview it locally | ||
- **[@dcl/sdk](packages/@dcl/sdk/README.md)**: Main SDK package for building Decentraland scenes using TypeScript/JavaScript | ||
- **[@dcl/ecs](packages/@dcl/ecs/README.md)**: Core Entity Component System (ECS) implementation with CRDT-based networking support | ||
- **[@dcl/react-ecs](packages/@dcl/react-ecs/README.md)**: React bindings for the ECS, providing a declarative way to build UIs using React's component model and JSX syntax | ||
- **[@dcl/js-runtime](packages/@dcl/js-runtime/README.md)**: TypeScript definitions for the Decentraland scene runtime environment. | ||
- **[@dcl/inspector](packages/@dcl/inspector/README.md)**: Visual editor and development tool for building Decentraland scenes | ||
- **[@dcl/sdk-commands](packages/@dcl/sdk-commands/README.md)**: CLI tools and commands for scene development, testing, and deployment | ||
- **[@dcl/playground-assets](packages/@dcl/playground-assets/README.md)**: Contains the built assets required by the Decentraland Playground | ||
|
||
1. Run `npx @dcl/sdk-commands init` on an empty folder. | ||
2. Preview it with `npm run start`! | ||
## Quick Start | ||
|
||
## Repository guide | ||
1. Clone the repository: | ||
|
||
This repository consists of the following components, packaged for the `nodejs`/`npm` ecosystem (find them under the respective subfolder in `packages`): | ||
```bash | ||
git clone https://github.com/decentraland/js-sdk-toolchain.git | ||
cd js-sdk-toolchain | ||
``` | ||
|
||
* `@dcl/react-ecs`: a framework to create scenes using the [React](https://reactjs.org) framework | ||
* `@dcl/sdk`: contains all the packages that a scene needs to work. | ||
* `@dcl/ecs`: an engine used to render things on screen | ||
* `@dcl/sdk-commands`: contains the command line interface | ||
* `@dcl/inspector`: Editor interface. | ||
2. Install dependencies and build all packages: | ||
|
||
And some internal or maybe useful packages if you're digging deeper into how the Decentraland runtime works: | ||
```bash | ||
make install | ||
make build | ||
``` | ||
|
||
* `@dcl/js-runtime`: the `js-runtime` contains the typings for the environment variables available in the sandboxed execution environment for scenes | ||
* `@dcl/playground-assets`: contains the files needed by the playground. <https://playground.decentraland.org/> | ||
## Development | ||
|
||
### Versioning notes | ||
### Building | ||
|
||
When `@dcl/sdk` is built, as it depends on new versions of `@dcl/ecs`, these are built first and `@dcl/sdk` includes the new versions. | ||
The project uses a Makefile to handle all build tasks: | ||
|
||
### ECS 6 dev support | ||
```bash | ||
# Build everything (including protobuf) | ||
make build | ||
|
||
The ECS 6 lives in the `6.x.x` branch, there will no longer be new features but it's available for fixes or patches. | ||
With a PR to `6.x.x`, you can test the build with the S3 publish, but it'll be necessary to create a release for propagating under `decentraland-ecs@latest`. | ||
# Clean build artifacts | ||
make clean | ||
|
||
### Updating golden files (.crdt) | ||
# Clean everything (including node_modules) and rebuild | ||
make deep-clean && make install && make build | ||
``` | ||
|
||
We use golden files to create snapshots for a series of test scenes. Most changes to the codebase impose a change in the amount of opcodes executed in the actual scene. We use a QuickJS virtual machine to benchmark how many opcodes are required. Even though this is not representative of the reallity of optimized JIT virual machines, it is a good approximation of the impact that the change would imposes on scene developers. | ||
### Testing | ||
|
||
To re-create these golden files, run `make build update-snapshots`. In some cases, this will generate some discrepancies with the clean environment used by the continuous integration we use (CircleCI). If you run into this issue, please run `make deep-clean-and-snapshot` to invalidate all cached calculations. **Be careful**: it will clean all local changes on your git [working tree](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddefworkingtreeaworkingtree). | ||
```bash | ||
# Run all tests | ||
make test | ||
|
||
### Release a new SDK Version | ||
# Run ECS tests | ||
make test-ecs | ||
|
||
After merging a PR to the main branch, we should wait until the [actions](https://github.com/decentraland/js-sdk-toolchain/actions/workflows/ci.yml) finish the process. | ||
# Run Inspector tests | ||
make test-inspector | ||
|
||
Once the actions completed successfuly, we have to create a new [Release](https://github.com/decentraland/js-sdk-toolchain/releases/new) from the main branch following this [version guideline](https://docs.decentraland.org/creator/releases/version-agreement/). | ||
# Run tests with coverage | ||
make test-coverage | ||
|
||
## Copyright info | ||
# Update test snapshots | ||
make update-snapshots | ||
``` | ||
|
||
This repository is protected with a standard Apache 2 license. See the terms and conditions in the LICENSE file. | ||
### Protobuf | ||
|
||
The project uses Protocol Buffers for type-safe communication. Protobuf files are automatically compiled during the build process. | ||
|
||
To manually rebuild protobuf files: | ||
|
||
```bash | ||
make proto | ||
``` | ||
|
||
## Release Process | ||
|
||
1. Merge changes to the main branch | ||
2. Wait for CI workflow completion in GitHub Actions | ||
3. Create a new Release following the [version guidelines](https://docs.decentraland.org/creator/releases/version-agreement/) | ||
|
||
## Testing Infrastructure | ||
|
||
### Snapshot Testing | ||
|
||
We use snapshot testing with golden files to track runtime performance impacts: | ||
|
||
- Snapshots measure QuickJS opcode execution for test scenes | ||
- Run `make build update-snapshots` to update golden files | ||
- For clean environment matching CI, use `make deep-clean-and-snapshot` | ||
> Note: This cleans all local changes in your git working tree | ||
### Running Tests | ||
|
||
```bash | ||
# All tests | ||
make test | ||
|
||
# Specific package | ||
make test-ecs | ||
make test-inspector | ||
``` | ||
|
||
## SDK Version Support | ||
|
||
### SDK7 | ||
|
||
The main branch contains SDK7, the current version. All new features and improvements target SDK7. | ||
|
||
### SDK6 Maintenance | ||
|
||
SDK6 is maintained in the `6.x.x` branch for critical fixes: | ||
|
||
- No new features are added | ||
- Only bug fixes and security patches | ||
- Create PRs against the `6.x.x` branch | ||
- Releases update the `decentraland-ecs` package | ||
|
||
## Troubleshooting | ||
|
||
If you encounter build issues: | ||
|
||
1. Clean the project and reinstall dependencies: | ||
|
||
```bash | ||
make clean && make install | ||
``` | ||
|
||
2. Rebuild everything and run tests: | ||
|
||
```bash | ||
make build && make test | ||
``` | ||
|
||
Common issues: | ||
|
||
- **Build failures**: Try `make clean && make install && make build` | ||
- **Test failures**: Run `make test` to see detailed errors | ||
- **Protobuf errors**: Run `make proto` to rebuild protocol buffers | ||
- **Package conflicts**: Delete `node_modules` and run `make install` again | ||
- **TypeScript errors**: Check package versions match in `package.json` files | ||
|
||
## Contributing | ||
|
||
1. Fork the repository | ||
2. Create your feature branch | ||
3. Commit your changes | ||
4. Push to the branch | ||
5. Create a Pull Request | ||
|
||
## Architecture Decisions | ||
|
||
For a deeper understanding of the architecture and design decisions: | ||
|
||
- [ADR-117: CRDT Protocol for Scenes](https://adr.decentraland.org/adr/ADR-117) - Details the scene state synchronization | ||
- [ADR-123: Schema and Serialization](https://adr.decentraland.org/adr/ADR-123) - Explains component data handling | ||
- [ADR-124: Implementing Flexbox-based UI](https://adr.decentraland.org/adr/ADR-124) - Describes the UI layout system | ||
- [ADR-125: User Interface Components](https://adr.decentraland.org/adr/ADR-125) - Covers the UI system architecture | ||
- [ADR-133: Scene Runtime Definition](https://adr.decentraland.org/adr/ADR-133) - Details how scenes are executed | ||
- [ADR-153: Transform SDK Component](https://adr.decentraland.org/adr/ADR-153) - Explains the core Transform component | ||
- [ADR-165: Component Declaration](https://adr.decentraland.org/adr/ADR-165) - Describes the ECS component system design | ||
- [ADR-237: SDK 7 Custom UI Components](https://adr.decentraland.org/adr/ADR-237) - Details the UI component system | ||
- [ADR-281: Items in Decentraland tooling](https://adr.decentraland.org/adr/ADR-281) - Explains the Items abstraction used across tools | ||
- [ADR-282: Decentraland Inspector](https://adr.decentraland.org/adr/ADR-282) - Details the Inspector's architecture and integration approaches | ||
|
||
For more ADRs, visit our [ADR repository](https://adr.decentraland.org/). | ||
|
||
## License | ||
|
||
Apache 2.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,117 @@ | ||
# ECS 7 | ||
# @dcl/ecs | ||
|
||
## Installing dependencies | ||
Run `make install`, this will run the `npm install` and other dependencies | ||
Core Entity Component System (ECS) package for Decentraland scenes. Implements a CRDT-based ECS architecture for networked scene state. | ||
|
||
## Building | ||
Run `make build` | ||
## Installation | ||
|
||
## Testing | ||
Run `make test`, you can also debug the test in VS code, selecting the launch `Jest current file` or just `Jest` (this will run all test) | ||
```bash | ||
npm install @dcl/ecs | ||
``` | ||
|
||
## Usage | ||
|
||
```typescript | ||
import { engine } from '@dcl/ecs' | ||
|
||
// Create entity | ||
const entity = engine.addEntity() | ||
|
||
// Define and add component | ||
const Health = engine.defineComponent(1, { | ||
current: Number, | ||
max: Number, | ||
regeneration: Number | ||
}) | ||
|
||
Health.create(entity, { | ||
current: 100, | ||
max: 100, | ||
regeneration: 1 | ||
}) | ||
|
||
// Create system | ||
engine.addSystem((dt: number) => { | ||
for (const [entity, health] of engine.mutableGroupOf(Health)) { | ||
if (health.current < health.max) { | ||
health.current = Math.min(health.max, health.current + health.regeneration * dt) | ||
} | ||
} | ||
}) | ||
``` | ||
|
||
## Technical Overview | ||
|
||
### Component Definition | ||
|
||
Components are defined with a unique ID and a schema. The schema is used to: | ||
|
||
- Generate TypeScript types | ||
- Create binary serializers/deserializers | ||
- Set up CRDT operations | ||
|
||
### CRDT Implementation | ||
|
||
The ECS uses CRDTs (Conflict-free Replicated Data Types) to enable deterministic state updates across multiple engine instances: | ||
|
||
- Component updates are CRDT operations with logical timestamps | ||
- Multiple engine instances can be synced by exchanging CRDT operations | ||
- Conflict resolution uses timestamps and entity IDs to ensure consistency | ||
- Binary transport format minimizes network overhead | ||
|
||
### Network Entities | ||
|
||
For multiplayer scenes, the `NetworkEntity` component marks entities that should be synchronized across peers: | ||
|
||
```typescript | ||
import { engine, NetworkEntity } from '@dcl/ecs' | ||
|
||
// Create a networked entity | ||
const foe = engine.addEntity() | ||
NetworkEntity.create(foe) | ||
|
||
// Components on this entity will be synced across peers | ||
Health.create(foe, { current: 100, max: 100, regeneration: 1 }) | ||
``` | ||
|
||
Each peer maintains its own engine instance. When using NetworkEntity: | ||
|
||
- The owner peer can modify the entity's components | ||
- Other peers receive read-only replicas | ||
- Updates are propagated through the network transport layer using CRDT operations | ||
|
||
Example transport message: | ||
|
||
```typescript | ||
{ | ||
entityId: number | ||
componentId: number | ||
timestamp: number | ||
data: Uint8Array // Serialized component data | ||
} | ||
``` | ||
|
||
### Performance Features | ||
|
||
- Zero-allocation component iteration | ||
- Dirty state tracking for efficient updates | ||
- Binary serialization for network transport | ||
- Batched component operations | ||
|
||
## Development | ||
|
||
```bash | ||
# Build | ||
make build | ||
|
||
# Test | ||
make test | ||
|
||
# Clean and reinstall | ||
make clean && make install | ||
``` | ||
|
||
## Documentation | ||
|
||
- [ECS Guide](https://docs.decentraland.org/creator/development-guide/sdk7/entities-components/) | ||
- [Component Reference](https://docs.decentraland.org/creator/development-guide/sdk7/components/) | ||
- [ADR-117: CRDT Protocol](https://adr.decentraland.org/adr/ADR-117) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
# @dcl/js-runtime | ||
|
||
TypeScript definitions for the Decentraland scene runtime environment. | ||
|
||
## Overview | ||
|
||
This package provides TypeScript type definitions for the Decentraland scene runtime environment. It includes: | ||
|
||
- Type definitions for Web APIs (fetch, WebSocket) | ||
- Auto-generated SDK/API type definitions | ||
- Basic runtime types (console, DEBUG flag) | ||
|
||
This package does not contain any JavaScript runtime code - it only provides TypeScript types and interfaces used by the SDK and scene developers. | ||
|
||
## Installation | ||
|
||
```bash | ||
npm install --save-dev @dcl/js-runtime | ||
``` | ||
|
||
## Development | ||
|
||
The types in this package are auto-generated during the monorepo's build process: | ||
|
||
```bash | ||
# From the root of the monorepo | ||
make build | ||
``` | ||
|
||
The generated types will be available in `apis.d.ts`. | ||
|
||
## Documentation | ||
|
||
- [Scene Development Guide](https://docs.decentraland.org/creator/development-guide/sdk7/scene-content/) | ||
- [Runtime API Reference](https://docs.decentraland.org/creator/development-guide/sdk7/runtime-api/) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# @dcl/playground-assets | ||
|
||
Assets package for the [Decentraland Playground](https://playground.decentraland.org/). | ||
|
||
## Overview | ||
|
||
This package contains the built assets required by the Decentraland Playground - an experimental web IDE that allows developers to: | ||
|
||
- Write and preview Decentraland scenes in real-time | ||
- Share code snippets with other developers | ||
- Test and iterate on scene code directly in the browser | ||
- View changes instantly in the integrated Decentraland web explorer | ||
|
||
Similar to [Babylon.js Playground](https://playground.babylonjs.com/), the Decentraland Playground provides an interactive environment for rapid prototyping and experimentation. | ||
|
||
## Development | ||
|
||
The assets in this package are automatically built during the monorepo's build process: | ||
|
||
```bash | ||
# From the root of the monorepo | ||
make build | ||
``` | ||
|
||
## Related Documentation | ||
|
||
- [Playground](https://playground.decentraland.org) |
Oops, something went wrong.