-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
VPL is not declarative #116
Comments
So, as long as you have don't-care positions, you can trigger two rules with the same conditions: Whatever the choice of semantics, the important thing is feedback to the programmer. My memory is rusty but I think checking for ambiguous rules is the same as constructing an Aho-Corasick automaton. |
The sensors are random number generators. Another reasonable semantics would be to give the priority to the sensor that has changed the most since the last clock tick |
That is interesting indeed. There are many possible semantics indeed, depending on whether we only consider the close/far thresholds (and hence the conditions), which works only when these thresholds cannot be changed, or whether we consider general conditions. Actually, when I think about it, I think that in VPL2, all true transition within the reactive mode are taken. @marvelous, is it correct? |
No, the implementation is single-threaded, only one row is triggered simultaneously. We could have one thread per row to support multiple concurrent row triggers, but the problem would resurface anyway when we add a mutable state variable: last write wins would be an obvious implementation choice. |
That sounds like an ugly hack. I much prefer to do an exhaustive check and raise a compilation error if not too runtime-intensive. An other option that seems simple to me is to do a runtime check, raise an aesl event and stop the robot when multiple transitions are possible, prompting the programmer to remove the ambiguity. |
BTW, do you have a realistic VPL1 program for this issue? |
For simplicity, let's invent a text syntax: Example 1
This is an attempt at a black line follower. The student's idea is to turn away from white, and to shine red if the robot gets lost on white. Question: if the robot gets lost, does it turn? Deterministically or stochastically? Example 2
This is an attempt at a wall avoider, where the student requires a consensus between proximity sensors to make a decision. If the threat of hitting the wall is high the robot swerves in place, otherwise it turns while advancing. If nothing is directly in front, then go forward at speed. Independently, shine red or yellow if a wall is detected. Questions: what color is shown when rule 2 is triggered? What speed is used when Possible answers: (red, yellow, or orange); (150||50, 200||200, or 175||125); (300||0, 0||300, 300||300, or 0||0) CommentsThese programs are syntactically correct and have a clear intention. They don't have to be the programs we would have written, or even correct; they are plausible programs that a student might try, and hope to learn from the observed behavior of the robot. We can't forbid these programs because of an arcane rule, that can't be explained to a 10-year-old. I can think of four semantics for these programs: deterministic using rule order, stochastic transition system, constraint solving over finite domains, analog (average between requested values). But really the question is, which semantics is best for helping students learn to program autonomous robots in a noisy world? |
Great examples! Let's ping @motib and @FrancescoMondada for input. |
David's examples use "don't care"s which are very much like variables in Prolog. I am in favor of choosing the Prolog semantics: execute the first clause that unifies, here, the first event-action pair whose event evaluates to true. It is easy to explain and deterministic, hence, repeatable, so you can debug the program. Any non-deterministic semantics would just confuse kids. There is lots of "noise" in the sensors and the environment, so I don't see the need for noise in the program semantics! |
It is hard that in the case of / _ 1 1 1 _ \ all sensors become true at the same time. Therefore I expect several lines to activate during the steps between / _ 000 _ \ and / _ 1 1 1 _ . This should better fit to what the programmer expects. If all three become 1 at the same moment, I agree with Moti that it should be deterministic: even the last or the first line satisfying the conditions should be executed. But the transition bit should be per line, and not per sensor. |
In example 2, the intent of the user seems to run 2 things concurrently: motor control and light control (the 5th line should even be split in two!). If the sensors go from / 1 0 0 0 0 \ to / 1 1 0 0 0 , should 150||50 or top(15,15,0) be executed? @davidjsherman says that both should be executed, @motib and @FrancescoMondada say that only 150||50 should be executed. Do I get this right? |
I agree with @FrancescoMondada that the noisiness of the environment means that many rules get their chance. But what is unfortunate is that a non-declarative semantics leads to a bias, which is what I tried to show in the examples. There isn't any way for the young programmer to define a fair strategy. In example 1, the lost robot always spins clockwise. In example 2, there is a bias towards turning right. And I fear that the programmer will have a hard time understanding why the lost robot never turns red. Actually, as a teacher I fear the trouble I will have explaining why to the programmer! |
@marvelous well spotted. I suppose we could say that each set of actuators (motors, LEDs, sound, IR comm) run in separate threads? |
@davidjsherman why the robot will never turn red? Because we have transition bits per sensor and not per line, and this a problem. |
@FrancescoMondada so you are arguing for having the conflict resolution between actions, not between lines? Both lines should run, but if they have conflicting actions only one of these actions is executed? Sorry if I misunderstood first. |
@marvelous yes, only conflict resolution between conflicting actions (and executing all of them will result in having the last only applied), it does not make sense to have conflicts between actions of different type |
The implementation will be simpler if we do the conflict resolution at the aesl variable level. The motor action block always sets both Using this last writer wins conflict resolution [insert distributed systems paper reference here], we can execute the action blocks in program order (last action block wins) or in reverse program order (first action block wins). |
Hello everyone, you keep talking about "program order" as if that is a thing, but the issue is named "VPL is not declarative". An implicit priority between rules is a cognitive bias that comes from sequential programming! Has a decision been made that you want to teach that to young programmers? |
@FrancescoMondada, I guess I don't understand which problem you mean in your comment? I understand that you want to trigger rules on the rising edge of sensor changes. Is the problem this strategy, or my example? Sorry to be so slow. Thanks for explaining. |
You are right, @davidjsherman! Going back to
The issue is that [ 0 0 ] would effect both 150||50 and 50||150. But it's not an issue that it would also effect top(31,0,0). Should the program detect this situation (before or during execution) and explain it to the user? So he would arrive to this solution:
(we won't tell him that the lights never turn back off) |
I am now confused as to the semantics of VPL. My intuition is that - while all event-action pairs are notionally parallel - in practice there is an endless loop that checks the events sequentially in textual order, jumping back to the first pair at the end of the program. Or are separate threads initiated for each pair? With interrupts to schedule them? Do multicore computers execute the threads in true parallelism?! If my intuition is correct, you teach that the pairs are executed in parallel, but if that leads to weird behavior, then the sequential model needs to be explained. Even in languages like Prolog that are designed to be declarative, the implementation is sequential. There were parallel logic programming languages, but they never made it outside the research labs. |
Let me add a clarification: I am using the (for me very important) concept of a "notional machine" proposed by Ben du Boulay in 1986. (For a modern survey see: Juha Sorva. Notional machines and introductory programming education. ACM Transactions on Computing Education (TOCE) 13(2), 2013.) This is the architecture that defines the semantics, even if it doesn't precisely reflect the implementation. Beginning students, in particular, should be presented with a very simple notional machine. That is why I am comfortable with a VPL notional machine that is fully parallel, even if the underlying implementation is different, and even if the notional machine has to modified in some situations. |
@motib and purely declarative logic languages that didn't even get that far! |
@motib I agreed that the semantics of the notional machine is paramount, we shouldn't define the meaning of the language in terms of what it easy to implement. When you say "fully parallel" do you mean that the sensor surveillance threads are running in parallel, or that every rule is notionally running in parallel (and thus will fire independently whenever its conditions are met)? |
@marvelous Yes, it would be nice to give advice for completing rules as you suggested. But I worry that it will lead to a combinatorial explosion. In Example 2, I made a cascade of all the front sensors. Wouldn't a complete solution require you to add 25 rules for all of the combinations? |
That is true. And indeed for this example, the solutions proposed by David, either averaging or sampling, are better. So let us think a bit whether one of these could be feasible. I think that averaging is more predictable than sampling, because the effect of the action will take place at the same time the events trigger. We could collect all rules that match, combine all their actions, and use the result to write the actuators. Note that this is well defined for colors and motors, but not so much for sound (we could average the generated melody, it would be fun, but I am not sure that this is what we want). Moreover, there is a problem because of the when semantics: if one rule becomes true after the others, this rule will dominate. If we average, the if semantics would be more correct, that is, apply all rules that are currently true, and combine their results. What we see is that the good solution seems to depend on the type of action. For continuous actions such as the wheels and the colors, an if semantics with some form of averaging or sampling seems the best. For sporadic actions such as sound, the when semantics with forbidden ambiguity at compile time seems the best. Note that we could also change the sound action to make it work well with if semantics, for instance if we just make it generate one note, whose pitch and intensity could be chosen by the programmer. The averaging of commands leads to an interesting, although a bit spooky, thinking about states. Do we want to have a probability distribution over states? That, allied with if semantics would lead to a closed-form and consistent semantics for the whole VPL, but I am afraid that the corresponding notional machine would be non-trivial to build. However, it might be very interesting from a scientific point of view, and actually we might want to make a study on this (before deploying it in the real world). But of course, we do not need to have a fully probabilistic state machine. We can consider that all lines that do not specify a state change vote for staying in the current state, and then the next state is the one for which there is more vote. @marvelous and @davidjsherman, what do you think about this analysis? |
All this make me think also that to understand what is going on the generated code is useful. Feedback from teacher also pointed often that VPL is good because you see the code. |
One elements that is bad for educational purposes with Thymio is that the API of the robot was designed to match the implementation, and not a clean interface for educational needs. For instance, there is no good educational reason why motors can be set with a variable while LEDs must be set through native functions. These are pure implementation details.
It is a point indeed. However, I am not sure how much looking at the code does really help the student, or whether it mostly help the teachers. In any case, it would be interesting to understand more in which context users look at the generated code. As said before, the code is not of optimal educational value because the API is too low level. |
Also, I think that teachers resort to looking at the generated at least partially because the VPL1 implementation has side effects and biases, compared to the notional machine. It would be interesting to see how users would react without such implementation issues. |
So, let us think how actions would react to multiple matching rules, assuming an if semantics for the events:
By voting, I mean that all active lines vote for the given action, and the result that has the majority wins, considering that "not executing the action" is also one option voted for. Note that the average semantics would correct the problem seen in VPL1, that lead to the use of when (see this explanation):
The following program will follow a track and stop when there is an obstacle in front. The mismatch problem between tap/clap (which happen only once) and the other events still remains. Btw, these two should probably be unified, as I do not remember a case when they were used together to denote different things. |
Is that meaning that if I have several event without this actions card set it is a "not executing the action" ? Most time with an if semantics it wil be the case.
Clap -> my robot start |
It is an interesting point. I was considering this for voting actions. Whether we should consider it for averaging actions is a question. I tend to say no, because for such an action (motors, color), the non-appearance of the block means "don't care" for that action, while for voting actions (music score, state change), the non-appearance of the block means "do not do it". |
This ugly hack is a compelling argument for not trying to second-guess the user intent! If the user first expresses a wrong solution:
This solution seems way more simple to explain than averaging tricks:
In this case, the proper solution is for the user to remove the ambiguity, like in example 1. Your points about the if semantics are interesting. From the code point of view, setting the motor speed is instantaneous, but from the user point of view, it is a continuous action. #69 I also think that state changes should trigger an event, which would break many VPL1.4 programs. But all these are discussions for other issues. |
I agree, and resolving the problem this way is totally compatible with averaging, if we want to also deal with example 2.
@motib, you implemented your tutorial both with if (for 1.3) and when (for 1.4) semantics. Which one would you say was more natural and led to the simplest notional machine?
I think that anyway we'll implement states differently than in VPL 1.4, so I'm not worried for that. |
The solution is compatible with averaging, but the averaging method will probably confuse the user when she hits the ambiguous cases before reaching the solution. |
Note that the problem expressed above that led to the choice of when semantics, can be solved now easily with if semantics because we allow conjuctions of events, as pointed out by @marvelous, regardless of the question of averaging or ambiguity removal. Regarding averaging, I think that there is an interesting and unique educational value of discussing the situation of multiple matching rules as pointed out by @davidjsherman. @miracet (Didier Roy), what do you think? |
First, I must admit that I haven't been following the details of this discussion for a while, but since @stephanemagnenat asked a question.... The programs in the tutorial were implemented before the change to "when". I'm not 100% sure, but I may have changed some of them to work with "when". It was quite unintuitive for simple programs, because "don't cares" (which is the default for prox sensors) had to be replaced with "not detected", requiring quite a few clicks. |
This is true, I agree. @motib in the case of your tutorial, would the example programs behave more naturally if there would be an averaging semantics in case of multiple line matches? I am thinking about the Braitenberg creatures for example. Please consider that conjunctions of multiple event blocks are available. |
I've thought a bit more about example 2, and I found a solution that is satisfying to me:
I constructed it by (imaging) trial and error, and it is still ambiguous. The assumption is that How can we help the user to build this program? By stopping the program when two actions are in conflict and showing that in the app. The user can then disambiguate the conflict, maybe with the help of a teacher. An other option for the user is to simply ignore the issue and restart the program. This is not a bad option, because the conflict might be due to an error in the physical setup. |
I looked at all the Braintenberg creatures. The only one that seems to be affected is "Driven": If an object is detected by the left sensor, the robot turns the right motor on and the It is ambiguous if both sensors detect an object, but I prefer ambiguity to trying to explain averaging, since the numerical values are not available to the kids in VPL like they are in Studio. Furthermore, variability between sensors and the sensitivity of the returned values to the slightest movement means that the average wouldn't be something you could easily demonstrate. |
Thanks @motib and @marvelous, both your examples are in support of run-time detection of ambiguity. In general, I am not a big fan of run-time error detection, especially for things like VPL, because:
However, if we are convinced that the understanding of whether a set of rules are ambiguous or not can only be resolved given a specific environment, then we have no choice, we have to do a run-time check, and stop the program and the robot in case of ambiguity, which I think is much better than taking the last line for instance. In that case, we should give a lot of attention to how we deal with the reporting of the error. For instance, we should leverage the fact that we have LEDs next to each sensor (thanks @FrancescoMondada, @mbonani, @riedo and others) and use these to tangibly report the source of the ambiguity. |
I am not sure it is annoying and it is in the loop of "trails and errors", that is in my sens pedagogical interesting in robotic.
probably yes
I am not convinced about that. |
For instance when Marie tried her prototype tutorial in two classes, going back and worth was a problem. Moreover, it is important to realise that we want to teach children an in-depth understanding of what the robot does, we want to let them build a proper notional machine. If we fail to do that, we basically produce a toy that has no educational value. Of course children need to explore by themselves, which is a key element of a constructionism approach, but this should be balanced with the ability of building a model of the program and doing self-guided search, not random search. Currently, as I have observed in the workshops I have given, the mismatch between the notional machine and the actual implementation leads to a mostly random search, especially if there are not enough assistants to personally tutor each child. |
I don't have real resources for studying notional models, but I spent an hour with my personal focus group and the completely unscientific anecdotal results are interesting. I proposed two ambiguous rules:
and I asked what they expected the robot to do in two situations. I didn't reveal that my secret intention was to express the rule "turn away from white". 1. On a white background. The consensus was, since both rules apply, the expectation was that the robot should oscillate between turning left and turning right. Nobody was sure how fast the oscillation should be, so we couldn't decide whether the robot zig-zagged, wiggled, or just moved forward slowly. I say consensus because some other options were discussed for a while. One idea that was bruited was that nothing could happen, since the robot can't decide. Another was that the robot could run around in a circle, although nobody had an idea about which direction it should be. 2. On the edge of a narrow black road bordered by white. In this case the clear expectation was that the robot will follow the road by zig-zagging from one edge to the next. |
So this is a sampling (maybe deterministic), rather than an averaging model.
Did the idea of an explicit error arise? |
They tried to imagine themselves play-acting the role of the robot, with different people calling orders. Do you try to be fair, do you listen to one group preferentially? The feeling seemed to be, if some call from the left and some call from the right, you try to satisfy both, and that means sidling forwards. There wasn't any feeling that the rules were inherently wrong, just that, under the circumstances, they were ambiguous. Indeed in the second situation the rules work fine. |
I don't want to inject my opinions in the language, because I am not in the target audience of programmers. But run-time checks remind me too much of Alan Cooper's warning against error messages: that whatever you say, the user will interpret them as saying "you are an idiot". So I would suggest setting a high standard for what constitutes "wrong", if indeed anything is. In what little I have observed (compared to all of you!) I have seen students very comfortable with the idea that the robot can act strangely, or in an unexpected way, and that this is a sign that they need to refine the program to make it better. So that is why I think that there aren't any programs that are inherently wrong, that every program has to do something, and that if that something is unexpected then an opportunity presents itself. |
I agree about disliking the idea of runtime error, in the same time I agree about @motib's argument against averaging. Indeed the need to have both averaging and voting as mixing rules also speaks against introducing averaging. Moreover for your focus group, the behaviour of alternating between the valid possibilities was considered a natural solution. Based on these, a reasonable solution could be to have a round-robin alternation between the matching solutions, using an if semantics. If this is sampled at a regular rate, it would produce the expected "hesitating" behaviour from the robot. This would work for all actions, knowing that there will be an uncertainty if two matching rules jump to different states, but this is ok if it is unbiased I think. Implementation-wise, this could be implemented by using the queue you introduced recently to collect matching rules, and then use a dispatch table. |
Following all these nice inputs, and after a short discussion with @marvelous, we think that it can be interesting to allow ambiguity with if semantics and use a sampling strategy between the matching lines, with a low sampling rate (1-4 Hz). This way, the robot would visibly hesitate between the matching lines, which could be a good way to let children understand the problematics of the ambiguity. There is also a clear notional machine, in case of several matching conditions, the robot applies the rules in turn. If necessary, we could highlight with additional visual hints (such as an animation of the LED circle to show that the robot is confused) the state of ambiguity. |
As pointed out by @davidjsherman, VPL is close to being declarative, but is not because when events conflicts (for instance with multiple configurations of distance sensors), the order of lines of appearance matters.
This problem has also been reported by teachers, and is actually one cause of the difficulty of using VPL to write non-trivial problems: the notional machine becomes difficult to create when the order matters.
It is interesting to think how this could be improved: currently, VPL forbids similar events or event configurations on two different lines. This could maybe be extended to forbid non-disjoint event conditions.
The text was updated successfully, but these errors were encountered: