Merge pull request #163 from node-red/flowdev

Reworked Flow Development Guide

Author: knolleary

_includes/toc-developing-flows.html

@@ -0,0 +1,19 @@
+<ul class="toc">
+    <li class="toc-expander"><div>V</div></li>
+    <li class="tocheader">
+        <ul>
+            <li class="toctitle active"><a href="/docs/developing-flows/">Developing Flows</a></li>
+            <li {% if page.url == "/docs/developing-flows/flow-structure" %}class="active"{% endif %}><a href="/docs/developing-flows/flow-structure">Flow structure</a></li>
+            <li {% if page.url == "/docs/developing-flows/message-design" %}class="active"{% endif %}><a href="/docs/developing-flows/message-design">Message design</a></li>
+            <li {% if page.url == "/docs/developing-flows/documenting-flows" %}class="active"{% endif %}><a href="/docs/developing-flows/documenting-flows">Documenting Flows</a></li>
+        </ul>
+    </li>
+</ul>
+<script>
+    $(function() {
+        var pageToc = $("<ul></ul>").appendTo(".toc li.active:not(.toctitle)");
+        $(".docs-content h3,.docs-content h4").each(function() {
+            $('<li id="toc-item-'+$(this).attr('id')+'"><a style="font-size: 0.9em; padding-left: 50px;" href="#'+$(this).attr('id')+'">'+$(this).text()+'</a></li>').appendTo(pageToc)
+        })
+    })
+</script>

_layouts/docs-developing-flows.html

@@ -0,0 +1,3 @@
+{% assign parent_url = '/docs/developing-flows/' %}
+{% assign parent_slug = 'developing flows' %}
+{% include docs-content.html %}

docs/developing-flows/documenting-flows.md

@@ -0,0 +1,115 @@
+---
+layout: docs-developing-flows
+toc: toc-developing-flows.html
+title: Documenting flows
+slug: documenting flows
+---
+
+In any programming language, a vital part of creating easy-to-maintain code is to ensure it is also well documented.
+
+Good documentation serves a number of purposes:
+
+1. Whilst everything may seem obvious as you are building a flow, your future self will thank you for providing some description of the details when you come back to it later.
+2. If you are sharing a flow with others, it will help them understand what it is doing and how it works.
+3. If a flow provides an external API you will want to document how that API should be used - what properties or parameters are expected.
+4. When you write documentation, the act of writing out the behaviour could well help you identify parts that could be improved.
+
+In a visual programming environment like Node-RED, the documentation can take a number of forms.
+
+- The flows can be read in the workspace to see the logical flow of events. You should make sure the purpose of each node is easily identified and that they are well laid out to minimise how much wires cross each other.
+- Groups can be used to identify discrete sections of the flows.
+- Moving commonly used parts into subflows can help reduce the visual complexity of the flow.
+- More complete documentation can be added at the node, group or tab level
+
+
+### Laying out flows
+
+The [flow structure](flow-structure) section of this guide looked at how to arrange the logical components of your flows. This section considers the visual appearance of the flow layout.
+
+The goal is to make it easy to follow the flow without having to jump around the workspace or have to follow multiple wires that cross each other and appear tangled.
+
+The approach that gives the greatest legibility is to keep each unit of processing on a single horizontal line wherever possible. The editor's default behaviour of snapping nodes to a grid on the tab helps keep them aligned.
+
+<div class="figure">
+    <img src="./images/node-arrangement.png" alt="Aligning flows in horizontal rows"/>
+    <p class="caption">Aligning flows in horizontal rows</p>
+</div>
+
+If there is a node that has more than one output port, aligning the branched flow vertically makes it easy to compare and contrast the flows.
+
+<div style="width: 600px" class="figure">
+  <img src="images/node-arrangement-sample-align.png" alt="Aligning branched flows">
+  <p class="caption">Aligning branched flows</p>
+</div>
+
+When a flow gets too long, arranging some nodes vertically can be used to good effect. In the following figure, some of the nodes are arranged vertically to imply a relationship between them. It is easier to understand the nature of the overall flow if it is visually obvious what smaller sections it is comprised of and how they relate to each other.
+
+<div class="figure">
+    <img src="./images/node-vertical-arrangement.png" alt="Vertically aligning logical segements of a long flow"/>
+    <p class="caption">Vertically aligning logical segements of a long flow</p>
+</div>
+
+In some cases, these smaller sections may be candidates for moving to subflows that will reduce the visual complexity of the flow. That is particular true if that smaller section could be reused elsewhere in the flows.
+
+### Naming nodes
+
+Most nodes have a `name` property that can be used to customise the label they display in the workspace. This should be used to properly label the key points of a flow.
+
+For example, if a Change node has a single rule that sets `msg.payload` to the current time, its default label will be `set msg.payload`. That helps somewhat, but it doesn't reveal the full purpose of the node. A name of `Get current time` would be much clearer.
+
+There is a balance to be considered here. The longer the label, the more space it needs in the flow. The shorter the label, the less information it can share.
+
+For some nodes, it might be appropriate to hide the label altogether to minimise the horizontal space it uses in the flow - giving more room to other nodes.
+
+Along with the label, nodes can also have a custom icon. For example, if you have a number of MQTT In nodes for different types of device, customising the icon to match the type of device could be helpful. This should be used with care as the icon is one of the main ways of identifying the type of a particular node
+
+Choosing good names for things applies just as much to the tabs and subflows used.
+
+It also very important for Link nodes. Without a name set, you have to use the Link node's internal ID when creating links between different tabs. That makes it hard to identify the right target node and mistakes can happen. If you consider the Link nodes as providing APIs between the different tabs, then a good choice of naming scheme will be needed. The names should clearly identify the start and end point of each flow.
+
+### Adding port labels
+
+If a node has multiple outputs it can be hard to follow the logic if it is not clear on what condition a message may be sent from a particular output.
+
+This is where adding port labels can help document the intended logic.
+
+For example, the Switch node provides default labels for its outputs that are shown when the mouse hovers over them. They can help quickly identify the purpose of each branch in the flow.
+
+Whilst the default labels may be sufficient in the context of the flow itself, it is also possible to customise labels to provide more detailed information.
+
+<div  class="figure">
+  <img src="images/node-output-labels.png" alt="Custom output labels on the Switch node's Appearance tab">
+  <p class="caption">Custom output labels on the Switch node's Appearance tab</p>
+</div>
+
+### Inline Comments
+
+The Comment node can be used to add inline comments to the flow - both the node's label, but also its description that will show in the Information sidebar when selected.
+
+By indenting the flows on the page, you can indicate an implied grouping of the different components.
+
+<div class="figure">
+  <img src="images/comment-nodes.png" alt="Documenting flows with the Comment node">
+  <p class="caption">Documenting flows with the Comment node</p>
+</div>
+
+### Grouping nodes
+
+A more explicit arrangement of the flows can be achieved by grouping related nodes together.
+
+The background colour of each group can also be used to highlight different types of group.
+
+<div class="figure">
+  <img src="images/grouping-nodes.png" alt="Grouping nodes">
+  <p class="caption"> Grouping nodes</p>
+</div>
+
+### Adding longer documentation
+
+All of the techniques discussed so far relate to the visual appearance of the flows. In order to add more in depth documentation, something more is needed.
+
+Every node, group and tab can have longer-form documentation added under the [Description tab in their edit dialog](/docs/user-guide/editor/workspace/nodes#editing-node-properties). This help can be formatted using Markdown and including lists, tables and links. This documentation is then displayed in the [Information sidebar](/docs/user-guide/editor/sidebar/info) when the item is selected.
+
+This longer format of documentation is useful where more explanation is needed of a flow's purpose, or some more complex logic needs to be described.
+
+It is also useful where a flow provides an external API of some sort - providing as much detail as it needed for other developers to use the API.

docs/developing-flows/flow-structure.md

@@ -0,0 +1,127 @@
+---
+layout: docs-developing-flows
+toc: toc-developing-flows.html
+title: Flow structure
+slug: flow structure
+---
+
+When you first start using Node-RED, you probably start adding all of your nodes to the same tab in the editor. You may import some example flows others have shared, or build prototype flows to test different things out.
+
+Over time, that can lead to a mess of nodes and wires that make it hard to find particular parts of the flow.
+
+Putting some thought into how to structure your flows at the start of any development project can help keep them organised and make them easier to maintain.
+
+The main method of organising flows in Node-RED is by separating them across multiple tabs within the editor. There are a few different strategies that can be used to do that.
+
+If you can identify separate logical components of your application, consider putting them on separate tabs.
+
+For a home-automation application, you could put the flow logic for each room on a separate tab to reflect the physical space. Or you may want to separate the flows based on function - so all lighting-related flows go on one tab and heating on another.
+
+If you are building an HTTP API backend, each tab could represent a separate type of resource the API accesses.
+
+The goal should be to make it easy to “read” an individual flow from start to finish. Keeping it on a single tab can help do that.
+
+Another consideration is whether you are working alongside other developers on the same Node-RED application. It is much easier to manage the merging of changes if they are on separate tabs. If you have developers with different roles or specialisations, consider how that may affect how your flows are organised.
+
+
+### Making reusable flows
+
+As you build your flows, you may find some common parts that you want to reuse in multiple places. You should avoid having multiple copies of those common parts spread across your flows as they become harder to maintain - you end up with multiple places to apply fixes and could easily overlook one.
+
+Node-RED provides two different ways of creating reusable flows - Links nodes and Subflows.
+
+<div style="width: 300px" class="figure align-right">
+  <img src="images/link-nodes.png" alt="Link nodes">
+  <p class="caption">Link nodes</p>
+</div>
+
+**Link nodes** let you create a flow that can jump between tabs in the editor - they add a virtual wire from the end of one flow to the start of another.
+
+<div style="clear:both"></div>
+
+<div style="width: 300px" class="figure align-right">
+  <img src="images/subflow.png" alt="Subflows">
+  <p class="caption">Subflows</p>
+</div>
+
+**Subflows** let you create a new node in the palette whose internal implementation is described as a flow. You can then add new instances of the subflow wherever you would a normal node.
+
+
+
+There are some important differences between the two approaches. Link nodes cannot be used in the middle of a flow, where messages are passed over the link and then return when the other flow completes. They can only be used to start or end a flow. They can also be connected to more than one other link node. This lets you pass messages out to multiple other flows, or have multiple flows pass messages into a single flow. They can be used within a single tab to help flows wrap across the workspace without having lots of wires crossing from right to left.
+
+Subflows appear as regular nodes so can be used at any point in a flow. However each instance of the subflow is independent of the others. Any flow context inside the subflow will be scoped to the individual instances. If the subflow creates a connection to a remote system, each instance will create its own connection.
+
+### Customising subflows
+
+When creating subflows, you may want to be able to customise their behaviour in some way. For example, changing what MQTT topic it publishes to.
+
+One pattern for doing that is by setting `msg.topic` on every message passed to the subflow. But that requires adding a Change node ange node in front of every subflow instance in order to set the desired value.
+
+An easier way for doing this is by using Subflow properties. These are properties that can be set on the subflow instance and appear as environment variables inside the subflow.
+
+In the MQTT example, you could first configure the node to publish to `${MY_TOPIC}`.
+
+<div class="figure">
+  <img src="images/mqtt-envvar.png" alt="MQTT topic set by environment variables">
+  <p class="caption">MQTT topic set by environment variables</p>
+</div>
+
+<div style="width: 400px" class="figure align-right">
+  <img src="images/subflow-envvar.png" alt="Adding a subflow property">
+  <p class="caption">Adding a subflow property</p>
+</div>
+
+Then add `MY_TOPIC` as a subflow property.
+
+<div style="clear:both"></div>
+
+When a user edits an individual instance they can then provide a custom value for `MY_TOPIC` for that instance.
+
+
+<div class="figure">
+  <img src="images/subflow-instance-envvar.png" alt="Customising a subflow instance property">
+  <p class="caption">Customising a subflow instance property</p>
+
+</div>
+
+
+This pattern can be applied to any node configuration field that lets you enter the value directly. It doesn’t currently work for fields that are exposed as checkboxes or other custom UI elements.
+
+### Managing state information
+
+Another consideration is how to manage any state information in your flows. For example, keeping a count of how many messages pass through a flow, or the current state of an external sensor.
+
+Node-RED provides the Context system for managing state within the runtime. The context can be scoped to the same tab, subflow or made available globally.
+
+If a piece of state information is only needed by nodes on a particular tab, you should use flow-scoped rather than global. You should also choose context variable names with care - make sure they are descriptive and easy to identify.
+
+Another option is to manage the state outside of Node-RED - such as using retained MQTT messages, or a database of some sort. Those options do add an external dependency to manage and aren’t as conveniently integrated as Context, but they can also be used alongside context and not as a complete replacement. For example, where you want to share the state information across multiple Node-RED instances, or in the case of MQTT, be able to trigger a flow whenever a value changes.
+
+
+### Customising flows for different platforms
+
+Environment variables can be used more widely within Node-RED to create flows that can be customised for different platforms without having to make manual changes.
+
+For example, you may have a flow that you plan to run on multiple devices, but each device should subscribe to its own unique MQTT topic.
+
+As with the subflow example above, you could configure the MQTT node to publish to ${MY_TOPIC} and then set that as an environment variable before running Node-RED. That allows those device-specific customisations to be maintained separately to the flows that should be common to all devices.
+
+The same approach can be used when the flows might run on different operating systems - where the path to a file used by the flows may be different depending on the OS.
+
+The Inject and Change nodes are able to access environment variables using either the "env" option in their TypedInput. The Function node can use the `env.get()` function.
+
+### Error handling
+
+Node-RED provides the Catch and Status nodes as ways of building flows that can respond to errors. For more information about how they can be used, refer to the [user guide](/docs/user-guide/handling-errors).
+
+As there is no direct visual association between a Catch node and the nodes it targets, you should consider how to position them in order to keep the flows readable.
+
+Placing them close to the parts of the flow they correspond to can help, but you should take care not cause your flows to become overcrowded.
+
+Another approach is to group all of the error handling flows below the main flow - making the 'good' path clearly distinct from the error paths.
+
+Giving your Catch nodes a clear name is also very important to help easily identify the scenarios they are intended to handle.
+
+Which ever approach you choose, try to be consistent across your different flows.
+

docs/developing-flows/images/comment-nodes.png

@@ -0,0 +1,34 @@
+---
+layout: docs-developing-flows
+toc: toc-developing-flows.html
+title: Introduction
+---
+
+Node-RED allows you to quickly start developing applications by dragging in nodes and
+wiring them together to create flows. This can be a great way to get started, but
+as flows grow over time, it can lead to applications that are harder to maintain.
+
+This guide provides some recommendations and best practices for how to create
+Node-RED flows that will be reusable, easier to maintain and more robust.
+
+This guide assumes that you are already familiar with the basic usage of Node-RED.
+If you are looking for more information about using Node-RED, the [User Guide](https://nodered.org/docs/user-guide/)
+and [Cookbook](https://cookbook.nodered.org/) are good resources to help you get
+started.
+
+
+
+### [Flow structure](flow-structure)
+
+This section looks at how you can organise your flows, strategies for splitting them
+into smaller, reusable components and how to customise them for different platforms.
+
+### [Message design](message-design)
+
+This section looks at how the design of messages can help create nodes and flows
+that work well together and are easier to maintain.
+
+### [Documenting flows](documenting-flows)
+
+All good code should have good documentation to match. This section looks at what
+tools and techniques Node-RED provides to help you document them.