Merge pull request SylphAI-Inc#103 from SylphAI-Inc/li

[Tutorial editing] Edit all raw tutorials to fix grammar errors and smooth the content

Author: Sylph-AI

docs/source/developer_notes/base_data_class.rst

@@ -300,7 +300,7 @@ The ``exclude`` parameter works the same across all methods.
 
 **DataClassFormatType**
 
-For data class format, we have :class:``core.base_data_class.DataClassFormatType`` along with ``format_class_str`` method to specify the format type for the data format methods.
+For data class format, we have :class:`DataClassFormatType<core.base_data_class.DataClassFormatType>` along with ``format_class_str`` method to specify the format type for the data format methods.
 
 .. code-block:: python
 

docs/source/developer_notes/component.rst

@@ -6,24 +6,30 @@ Component
 
 ..    `Li Yin <https://github.com/liyin2015>`_
 
-What you will learn?
+.. What you will learn?
+
+.. 1. What is ``Component`` and why is it designed this way?
+.. 2. How to use ``Component`` along with helper classes like ``FunComponent`` and ``Sequential``?
+
+
+:ref:`Component<core-component>` is to LLM task pipelines what `nn.Module` is to PyTorch models.
+It is the base class for components such as ``Prompt``, ``ModelClient``, ``Generator``, ``Retriever`` in LightRAG.
+Your task pipeline should also subclass from ``Component``.
+
 
-1. What is ``Component`` and why is it designed this way?
-2. How to use ``Component`` along with helper classes like ``FunComponent`` and ``Sequential``?
 
 Design
 ---------------------------------------
- :ref:`Component<core-component>` is to LLM task pipelines what ``nn.Module`` is to PyTorch models.
 
-It is the base class for components, such as ``Prompt``, ``ModelClient``, ``Generator``, ``Retriever`` in LightRAG.
-Your task pipeline should subclass from ``Component`` too. Instead of working with ``Tensor`` and ``Parameter`` to train models with weights and biases, our component works with any data, ``Parameter`` that can be any data type for LLM in-context learning, from manual to auto prompt engineering.
-We name it differently to avoid confusion and also for better compatibility with `PyTorch`.
+Different from PyTorch's nn.Module, which works exclusively with Tensor and Parameter to train models with weights and biases, our component can work with different types of data, from a string or a list of strings to a list of :class:`Document<core.types.Document>`.
+
+..  `Parameter` that can be any data type for LLM in-context learning, from manual to auto prompt engineering.
 
 
+Here is the comparison of writing a PyTorch model and a LightRAG task pipeline.
 
-Here is the comparison of writing a PyTorch model and a LightRAG task component.
 
-.. grid:: 2
+.. grid:: 1
     :gutter: 1
 
     .. grid-item-card::  PyTorch
@@ -65,28 +71,49 @@ Here is the comparison of writing a PyTorch model and a LightRAG task component.
                 def call(self, query: str) -> str:
                     return self.doc(prompt_kwargs={"input_str": query}).data
 
+As the fundamental building block in LLM task pipelines, the component is designed to serve five main purposes:
+
+1. **Standardize the interface for all components.**
+   This includes the `__init__` method, the `call` method for synchronous calls, the `acall` method for asynchronous calls, and the `__call__` method, which by default calls the `call` method.
 
-As the foundamental building block in LLM task pipeline, the component is designed to serve five main purposes:
+2. **Provide a unified way to visualize the structure of the task pipeline**
+   via the `__repr__` method. Subclasses can additionally add the `_extra_repr` method to include more information than the default `__repr__` method.
 
-1. **Standarize the interface for all components.** This includes the `__init__` method, the `call` method for synchronous call, the `acall` method for asynchronous call, and the `__call__` which in default calls the `call` method.
-2. **Provide a unified way to visualize the structure of the task pipeline** via `__repr__` method. And subclass can additional add `_extra_repr` method to add more information than the default `__repr__` method.
-3. **Tracks, adds all subcomponents and parameters automatically and recursively** to assistant the building and optimizing process of the task pipeline.
-4. **Manages the states and serialization**, with `state_dict` and `load_state_dict` methods in particular for parameters and `to_dict` method for serialization of all the states fall into the component's attributes, from subcomponents to parameters, to any other attributes of various data type.
-5. **Make all components configurable from using `json` or `yaml` files**. This is especially useful for experimenting or building data processing pipelines.
+3. **Track and add all subcomponents and parameters automatically and recursively**
+   to assist in the building and optimizing process of the task pipeline.
 
-These features are key to keep LightRAG pipeline transparent, flexible, and easy to use.
+4. **Manage the states and serialization**,
+   with `state_dict` and `load_state_dict` methods specifically for parameters, and the `to_dict` method for serialization of all states within the component's attributes, from subcomponents to parameters, to any other attributes of various data types.
+
+5. **Make all components configurable using `json` or `yaml` files**.
+   This is especially useful for experimenting or building data processing pipelines.
+
+These features are key to keeping the LightRAG pipeline transparent, flexible, and easy to use.
 By subclassing from the `Component` class, you will get most of these features out of the box.
 
 
+.. As the foundamental building block in LLM task pipeline, the component is designed to serve five main purposes:
+
+.. 1. **Standarize the interface for all components.** This includes the `__init__` method, the `call` method for synchronous call, the `acall` method for asynchronous call, and the `__call__` which in default calls the `call` method.
+.. 2. **Provide a unified way to visualize the structure of the task pipeline** via `__repr__` method. And subclass can additional add `_extra_repr` method to add more information than the default `__repr__` method.
+.. 3. **Tracks, adds all subcomponents and parameters automatically and recursively** to assistant the building and optimizing process of the task pipeline.
+.. 4. **Manages the states and serialization**, with `state_dict` and `load_state_dict` methods in particular for parameters and `to_dict` method for serialization of all the states fall into the component's attributes, from subcomponents to parameters, to any other attributes of various data type.
+.. 5. **Make all components configurable from using `json` or `yaml` files**. This is especially useful for experimenting or building data processing pipelines.
+
+.. These features are key to keep LightRAG pipeline transparent, flexible, and easy to use.
+.. By subclassing from the `Component` class, you will get most of these features out of the box.
+
+
 Component in Action
 ---------------------------------------
 
-.. Transparency
-.. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+
 
 In this note, we are creating an AI doctor to answer medical questions.
 Run the ``DocQA`` on a query:
 
+
 .. code-block:: python
 
     doc = DocQA()
@@ -133,6 +160,7 @@ Configure from file
 As the above example shows, we added subcomponent via attributes.
 We can also use methods to add more subcomponnents or parameters.
 
+
 .. code-block:: python
 
     from lightrag.core.parameter import Parameter
@@ -141,8 +169,12 @@ We can also use methods to add more subcomponnents or parameters.
     # list all parameters
     for param in doc.named_parameters():
         print(param)
-    # output
-    # ('demo', Parameter: demo)
+
+The output:
+
+.. code-block::
+
+    ('demo', Parameter: demo)
 
 You can easily save the detailed states:
 
@@ -152,21 +184,25 @@ You can easily save the detailed states:
 
     save_json(doc.to_dict(), "doc.json")
 
+To add even more flexibility, we provide :class:`FunComponent<core.component.FunComponent>` and :class:`Sequential<core.container.Sequential>` for more advanced use cases.
 
-To adds even more flexibility, we provide :class:`core.component.FunComponent` and :class:`core.component.Sequential` for more advanced use cases.
 
 
 Searalization and deserialization
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-We provide ``is_pickable`` method to check if the component is pickable.
-And any of your component, it is a good practise to ensure it is pickable.
+We provide the ``is_pickable`` method to check if the component is pickable.
+It is good practice to ensure that any of your components are pickable.
+
+
+
+
 
 FunComponent
 --------------
- Use :func:`core.component.fun_to_component` as a decorator to convert any function to a Component with its unique class name.
+ Use :func:`fun_to_component<core.component.fun_to_component>` as a decorator to convert any function to a Component with its unique class name.
 
-:class:`core.component.FunComponent` is a subclass of :class:`core.component.Component` that allows you to define a component with a function.
+:class:`FunComponent<core.component.FunComponent>` is a subclass of :class:`Component<core.component.Component>` that allows you to define a component with a function.
 You can directly use this class as:
 
 .. code-block:: python
@@ -180,16 +216,21 @@ You can directly use this class as:
     print(fun_component(1))
     print(type(fun_component))
 
-    # output:
-    # 2
-    # <class 'core.component.FunComponent'>
+The printout:
+
+.. code-block::
 
+    2
+    <class 'core.component.FunComponent'>
 
-We also have :func:`core.component.fun_to_component` to convert a function to a FunComponent via decorator or directly call the function.
+
+
+We also have :func:`fun_to_component<core.component.fun_to_component>` to convert a function to a `FunComponent` via a decorator or by directly calling the function.
 This approach gives you a unique component converted from the function name.
 
 Via direct call:
 
+
 .. code-block:: python
 
     from lightrag.core.component import fun_to_component
@@ -198,12 +239,17 @@ Via direct call:
     print(fun_component(1))
     print(type(fun_component))
 
-    # output:
-    # 2
-    # <class 'lightrag.core.component.AddOneComponent'>
+The output:
+
+.. code-block::
+
+    2
+    <class 'lightrag.core.component.AddOneComponent'>
 
 
-Via decorator will be even more convenient to have a component from a function:
+
+
+Using a decorator is an even more convenient way to create a component from a function:
 
 .. code-block:: python
 
@@ -220,8 +266,12 @@ Via decorator will be even more convenient to have a component from a function:
 
 Sequential
 --------------
-We have :class:`core.component.Sequential` class to PyTorch's ``nn.Sequential`` class. This is especially useful to chain together components in a sequence.  Much like the concept of ``chain`` or ``pipeline`` in other LLM libraries.
-Let's put the FunComponent and DocQA together in a sequence:
+
+
+
+We have the :class:`Sequential<core.container.Sequential>` class, which is similar to PyTorch's ``nn.Sequential`` class.
+This is especially useful for chaining together components in a sequence, much like the concept of ``chain`` or ``pipeline`` in other LLM libraries.
+Let's put the `FunComponent`` and `DocQA`` together in a sequence:
 
 .. code-block:: python
 
@@ -236,9 +286,12 @@ Let's put the FunComponent and DocQA together in a sequence:
     query = "What is the best treatment for headache?"
     print(seq(query))
 
-We automatically enhance users' queries before passing them to the DocQA component.
+We automatically enhance users' queries before passing them to the `DocQA` component.
 The output is:
 
+
+
+
 .. code-block::
 
     1. Over-the-counter pain relievers like acetaminophen, ibuprofen, or aspirin
@@ -269,4 +322,4 @@ The structure of the sequence using ``print(seq)``:
    - :func:`core.component.fun_to_component`
 
 
-We will have more advanced use cases in the upcoming tutorials.
+We will cover more advanced use cases in the upcoming tutorials.

docs/source/developer_notes/index.rst

@@ -41,11 +41,30 @@ We have a clear :doc:`lightrag_design_philosophy`, which results in this :doc:`c
    class_hierarchy
 
 
+Introduction
+-------------------
+
+
+:ref:`Component<core-component>` is to LLM task pipelines what `nn.Module` is to PyTorch models.
+An LLM task pipeline in LightRAG mainly consists of components, such as a `Prompt`, `ModelClient`, `Generator`, `Retriever`, `Agent`, or any other custom components.
+This pipeline can be `Sequential` or a Directed Acyclic Graph (DAG) of components.
+A `Prompt` will work with `DataClass` to ease data interaction with the LLM model.
+A `Retriever` will work with databases to retrieve context and overcome the hallucination and knowledge limitations of LLM, following the paradigm of Retrieval-Augmented Generation (RAG).
+An `Agent` will work with tools and an LLM planner for enhanced ability to reason, plan, and act on real-world tasks.
+
 
+Additionally, what shines in LightRAG is that all orchestrator components, like `Retriever`, `Embedder`, `Generator`, and `Agent`, are model-agnostic.
+You can easily make each component work with different models from different providers by switching out the `ModelClient` and its `model_kwargs`.
+
+
+We will introduce the libraries starting from the core base classes, then move to the RAG essentials, and finally to the agent essentials.
+With these building blocks, we will further introduce optimizing, where the optimizer uses building blocks such as Generator for auto-prompting and retriever for dynamic few-shot in-context learning (ICL).
 
 Building
 -------------------
 
+
+
 Base classes
 ~~~~~~~~~~~~~~~~~~~~~~
 Code path: :ref:`lightrag.core <apis-core>`.