more writing imported from elsewhere

Author: ajbouh

writing/milestone-0/OVERVIEW.md

@@ -13,39 +13,68 @@ We have plans for single-, double-, and quad-accelerator expansion boxes that ca
 Who It's For
 Initially, this project will serve hackers who want to build and collaborate with one another on local and intelligent systems. 
 
+### Who is it for? What are some sample situations?
+
+A1. Hackers (working on an open source software/hardware project)
+
+A2. Students (in grad school doing research, with international and multilingual collaborators)
+
+A3. Small and mid-sized businesses (a regional distributor)
+
+
 ### What It Is
 
 #### What you get in the mail:
-- Substrate System 0 is an easy-to-assemble kit AI console for home and work.
+- Substrate System 0 is an easy-to-assemble AI computer kit for home and work.
 
 #### Once built, it runs:
 - SubstrateOS, an operating system that boots from a USB drive and schedules AI models on internal and external consumer GPUs.
 
 #### Once booted, it can:
-- M0.1. Join in on real-world conversations with your other devices.
+- M0.1. Join in on real-world conversations via your other devices. [A1] [A2] [A3]
+    A1. Planning meeting
+    A2. Lecture
+    A3. Sales meeting, customer call
 
-- M0.2. Take meeting and lecture notes so you can pay full attention to the speakers. [Liv]
+- M0.2. Take meeting and lecture notes so you can pay full attention to the speakers. [Liv] [A1] [A2] [A3]
+    A1. Meeting minutes, next steps
+    A2. Lecture notes
+    A3. ?
 
-- M0.3. Identify and link to more info on people, events, and cited documents. [Liv]
+- M0.3. Identify and link to more info on people, events, and cited documents. [Liv] [A1] [A2]
+    A1. ?
+    A2. Important dates for the course; references to other talks, textbooks, papers, textbooks
 
-- M0.4. Keep an at-hand, ever-present record of conversation, tasks, and specific contributions. This captures good ideas as they happen, foster new connections between them, and helps maintain understanding between collaborators. [Matt] [Liv]
+- M0.4. Keep an at-hand, ever-present record of conversation, tasks, and specific contributions. This captures good ideas as they happen, foster new connections between them, and helps maintain understanding between collaborators. [Matt] [Liv] [A1] [A2] [A3]
+    A1. Feature suggestions, observations, 
+    A2. ?
+    A3. ?
 
-- M0.5. Eliminate language barriers by translating between any number of supported languages in real-time, across speech and text.
+- M0.5. Eliminate language barriers by translating between any number of supported languages in real-time, across speech and text. [A2] [A3]
+    A2. ?
+    A3. ?
 
-- M0.6. Respond only when actively addressed directly or indirectly, so using its name in a sentence does not result in false positives, but people also don't need to keep using its name explicitly to continue a conversation
+- M0.6. Respond only when actively addressed directly or indirectly, so using its name in a sentence does not result in false positives, but people also don't need to keep using its name explicitly to continue a conversation.
 
 - M0.7. Self-assess its reasoning capabilities so it knows what it can do and how well it performs relative to expectations.
 
-- M0.8. Cast wirelessly to nearby displays. [?Jeff]
-
-- M0.9. Help draft comparative analyses of ideas shared in both documents and live interactions.
-
-- M0.10. Allow you to build and contribute new functionality others can immediately adopt. [?Jeff]
+- M0.8. Cast wirelessly to nearby displays. [?Jeff] [A1] [A2] [A3]
+    A1. ?
+    A2. ?
+    A3. ?
 
-- M0.11. Provide alternative heuristics for approaching a problem (but not the answer). For example: "have you considered breaking the problem down into smaller chunks"; "have you a different representation for the problem?" (the agent tries to make the person think.) [Yoshiki]
+- M0.9. Help draft comparative analyses of ideas shared in both documents and live interactions. [A1] [A2]
+    A1. ?
+    A2. ?
 
-- M0.12. Track, integrate and synthesize a set of ongoing concerns or questions with media collected, notes, links saved [Matt]
+- M0.10. Allow you to build and contribute new functionality others can immediately adopt. [?Jeff] [A1] [A2]
+    A1. ?
+    A2. ?
 
-#### Collaborators can go beyond this and
-Build their own services and add them to the system
+- M0.11. Provide alternative heuristics for approaching a problem (but not the answer). For example: "have you considered breaking the problem down into smaller chunks"; "have you a different representation for the problem?" (the agent tries to make the person think.) [Yoshiki] [A1] [A2] [A3]
+    A1. ?
+    A2. ?
+    A3. ?
 
+- M0.12. Track, integrate and synthesize a set of ongoing concerns or questions with media collected, notes, links saved [Matt] [A1]
+    A1. ?

writing/org/OVERVIEW.md

@@ -119,13 +119,15 @@ We support and collaborate closely with young and upcoming projects like Llamafi
 
 Voice-based assistant systems like Amazon's Alexa, Apple's Siri, and Google Home.
 
-❌ None are open, hackable, or extensible. None have research platforms.
+    ❌ None are open, hackable, or extensible. None have research platforms.
+
 Copilot-like experiences from GitHub, Microsoft, and Google.
 
-❌  None are truly local and bespoke tailored intelligence for your unique needs.
+    ❌ None are truly local and bespoke tailored intelligence for your unique needs.
+
 AI SaaS offerings from OpenAI, Anthropic, and Google.
 
-❌  All of them make users the product for massive data mining operations.
+    ❌ All of them make users the product for massive data mining operations.
 
 ---
 

writing/tech/OVERVIEW.md

@@ -0,0 +1,252 @@
+# Meet Substrate
+
+---
+
+### problem: the state of intelligence
+
+When you read a book or play an instrument, most of the benefit accrues to you, the reader. 
+With AI, the more intelligence is accumulated, the more it benefits the technology owner, which in this case, is not the user.
+
+AI has moved fast over the last year. Intelligence is on a path to becoming "too cheap to meter". 
+
+We are simultaneously accumulating and expanding intelligence at a speed like never before, and also consolidating it all in the hands of the technology makers. 
+
+That’s the equivalent of your music knowledge benefiting the company that makes the violin.
+
+At any moment, the company can revoke the knowledge from you, use it for its own benefit, change the rules of engagement, or alter the fundamental architecture of the whole underlying system.
+
+---
+
+### solution: an architecture for intelligence
+
+If you go back to our instrument example:
+
+Imagine as you are using next generation computing systems, your own platforms themselves become more intelligent. These systems can help us better understand each other, better understand the world around us, and better navigate the immense challenges before us. 
+
+The systems we are building return leverage to the end users.  
+
+To do that, we need to turn the creation of AI into an even more open, global and multiplayer activity.
+
+We need it to be both open and accessible.
+
+We need an architecture for intelligence, now. 
+
+Substrate is a new kind of technology organization dedicated to the sole mission of inventing the architecture of intelligence, and developing the technology stack to enable it.
+
+---
+
+### what kind of architecture?
+
+[TODO insert photo]
+
+---
+
+### what is substrate?
+
+A new computer console that's designed for running AI locally. It runs a new OS and has a open, extensible suite of functionality.
+
+A radically accessible project, where we teach the world not just how to use the AI and console itself, but how build on it, how to hack it, and how to (re)make it themselves.
+
+---
+
+### why locally?
+
+It's where the people are!
+
+Lots of local devices and sensors → HCI possibilities are awesome.
+
+When users pay for electricity, there is less need to "monetize" them.
+
+Cheaper, more fun, better alignment with end users.
+
+---
+
+### why open and extensible?
+
+Invites creativity and experimentation.
+
+This allows value accrue to the end users.
+
+Ensures that such a powerful technology can't be captured by any single entity.
+
+---
+
+### why a console?
+
+AI has extreme memory/compute/bandwidth requirements
+
+Phone/tablet/laptop/desktop based designs are limited by space, bandwidth, cooling, and power
+
+But high-end (enterprise grade) hardware is expensive and hard to buy
+
+For things to keep moving, we need a common dev target with a minimum spec
+
+Many similarities to the pre-Xbox and pre-Playstation gaming ecosystem
+
+---
+
+### why an OS?
+
+Optimize performance (per $) and (per watt)
+
+Sub-second model loading requires tuning hardware, drivers, filesystems
+
+Smaller support matrix means higher quality
+
+Initial experience, upgrades, debugging are all easier
+
+---
+
+### substrate: a disclaimer
+
+Under heavy development
+
+This is the first technical presentation about our work
+
+Here be dragons
+
+---
+
+### substrate: intro
+
+Services are accessible via HTTP
+
+UI (if any) is via browser technologies
+
+Everything runs on device and is accessed via LAN or VPN
+
+Access to the internet is restricted
+
+---
+
+### substrate: real-world challenges
+
+The ecosystem is moving quickly (version conflicts) → containers*
+
+Model weights are large and a bad fit for container layers → resourcedirs
+
+Different flavors for each model (size, quantization, precision) → late binding config
+
+Service discovery & orchestration → gateway
+
+Stateful persistence → spaces
+
+SOTA keeps moving, so lots of model churn → blackboard
+
+---
+
+### substrate: model dev experience
+
+Build a container image with an HTTP server listening on $PORT
+
+*optionally*
+
+Declare the resource directories you need (like model weights from huggingface)
+
+Declare environment variables based on CUDA memory, RAM
+
+Declare the request patterns your service can satisfy
+
+---
+
+### substrate: accessing services
+  
+|   |user|from UI|from backend|
+|---|----|-------|------------|
+|directly|/gw/whisper/|/gw/whisper/|http://substrate/gw/whisper/|
+|indirectly|❌|/bb|http://substrate/bb|
+
+--- 
+
+### substrate: an example service
+
+[TODO] include from elsewhere
+
+---
+
+### substrate: meet blackboard
+
+A response to a number of challenges: adding new models, easier composition, a path to auto optimization, self improvement and self extension
+
+An old idea, new again
+
+Based on ideas from tuple space and Prolog (unification)
+
+Easy to get started with, can be a "drop in" replacement for REST
+
+---
+
+### substrate: meet blackboard
+  
+|   |blackboard approach|traditional approach|
+|---|-------------------|--------------------|
+1 request|≥ 0 responses|1 response
+routing matches on|full request and response|HTTP method, URL parts
+service discovery|builtin|env vars, DNS, hardcoded
+shadow traffic|builtin|ad hoc
+request tracing|builtin|ad hoc
+
+---
+
+### substrate: an example (before blackboard)
+
+[TODO] include from elsewhere
+
+---
+
+### substrate: a service example (after blackboard)
+
+[TODO include from elsewhere]
+
+---
+
+### status update
+
+We have a team of folks working on this project
+
+Milestone 0 is in progress
+
+System 0 is in progress
+
+---
+
+### follow along
+
+github.com/ajbouh/substrate
+[email protected]
+
+---
+
+### Questions?
+
+---
+
+### appendix
+
+---
+
+### how blackboard works
+
+---
+
+### the promise of PCIe
+
+---
+
+### how bridge works
+
+---
+
+### how to make your own os
+
+---
+
+### substrate: blackboard eases model testing
+
+[TODO] show a message sequence diagram for adding a second model
+
+---
+
+### substrate: blackboard and self improvement
+
+[TODO] show a message sequence diagram for self improvement