API to retrieve git specific build information

[philwhln]

We're using lambdacd with lambdacd-git to build any git branch or tag. This results in build step variables for git changed-ref, git revision (commit id) and git commits. These are very useful for tracking and monitoring build information across branches. Especially branches such as master. We're keen to know what the latest state of a given git branch is, but unfortunately with the current API this is not possible. We understand that lambdacd is decoupled so that git is not a first class citizen, but rather just something that may be part of a build pipeline. For us though, we do want git information and particularly git branch information be a first class citizen. The current API for build history does not include any git information, so even augmenting what it currently available from /api/builds is not possible.

Here's what I have implemented as a short-term solution for our project to solve this, but it requires some duplication of lambdacd code, which isn't optimal.
https://gist.github.com/philwhln/5334e7399405eb9f3b8f

Any thoughts and feedback on how this might be better implemented or ways in which lambdacd might be modified to better support this kind of extensibility would be much appreciated.

[flosell]

As I understand you, this API is meant to provide data for a separate dashboard, not the the existing LambdaCD UI.
In this case, I think its reasonable for that to have its own backend. The existing REST API is built with the same principle, to provide the the features necessary for the LambdaCD UI. Think of it as a "Backend for Frontend", to abuse the latest buzzword :)

In fact, I don't need LambdaCD Core to be the tool that does it all, I'm trying to build a basis others can build on. So different interfaces for different use cases are even encouraged.

But you are right, you shouldn't be forced to duplicate so much of LambdaCDs code to achieve that. I'll look into the code a bit more in the next days to see what could be made public to improve that.

In the longer run, I have some ideas to make this more elegant. Maybe a concept like "build metadata" build steps (like git) could write into but also to capture things like build times.

[philwhln]

As I understand you, this API is meant to provide data for a separate dashboard, not the the existing LambdaCD UI.

Yes, this is information we're interested in generally, whether through UI or other integrations.

In fact, I don't need LambdaCD Core to be the tool that does it all, I'm trying to build a basis others can build on. So different interfaces for different use cases are even encouraged.

+1

But you are right, you shouldn't be forced to duplicate so much of LambdaCDs code to achieve that. I'll look into the code a bit more in the next days to see what could be made public to improve that.

Thanks!

In the longer run, I have some ideas to make this more elegant. Maybe a concept like "build metadata" build steps (like git) could write into but also to capture things like build times.

Yes, I was thinking along these long lines, so that a pipeline can bubble up key information.

[flosell]

I also created a new issue with my thoughts on build metadata (#86) and made some of the common functions public.
Here's what your code would now look like: https://gist.github.com/flosell/fa007b800878b9f08409#file-pipeline_state-clj

What do you think?

[philwhln]

What do you think?

Looks great!

[philwhln]

btw, I tested out your changes and works well.

[flosell]

Thanks for the feedback!