Skip to content

Commit 0756562

Browse files
committed
Agent: automate generation of Kotlin bindings
Until now, we have been manually writing Kotlin bindings for the agent protocol. This worked fine when only a small number of people worked on the protocol and the protocol was small, but has become increasingly problematic as more people are actively working on the JetBrains plugin and the size of the Agent protocol has grown. For example, if you make a mistake in the bindings, you may end up spending a long time debugging why something is not working because the error messages can be cryptic. This PR adds a new script to automatically generate Kotlin bindings so that we no longer need to rely on manually written bindings. Simplified, the script works like this: - Index the codebase with a fork of scip-typescript. This fork uses an extension of SCIP that adds `SymbolInformation.signature` (more details here sourcegraph/scip#231). - Use the structured type signatures from the SCIP index, translate it into Kotlin code, and report errors if something in the protocol has no clean translation to Kotlin. - Add the generated code to the Cody repo so that we can closely track what Cody PRs are influencing the protocol, and how they're influencing it.
1 parent 1774aa6 commit 0756562

File tree

174 files changed

+9205
-97
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

174 files changed

+9205
-97
lines changed

Diff for: .github/workflows/agent-bindings.yml

+36
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
name: agent-bindings
2+
on:
3+
push:
4+
paths:
5+
- '**.ts'
6+
- '**.tsx'
7+
- '**.js'
8+
9+
jobs:
10+
kotlin:
11+
if: github.repository == 'sourcegraph/cody'
12+
runs-on: ubuntu-latest
13+
steps:
14+
- uses: actions/checkout@v3
15+
- uses: pnpm/action-setup@v2
16+
id: pnpm-install
17+
with:
18+
version: 8.6.7
19+
run_install: false
20+
- name: Get pnpm store directory
21+
id: pnpm-cache
22+
shell: bash
23+
run: |
24+
echo "STORE_PATH=$(pnpm store path)" >> $GITHUB_OUTPUT
25+
- uses: actions/cache@v3
26+
name: Setup pnpm cache
27+
with:
28+
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
29+
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
30+
restore-keys: |
31+
${{ runner.os }}-pnpm-store-
32+
- run: pnpm install --frozen-lockfile
33+
- run: ./agent/scripts/generate-agent-kotlin-bindings.sh
34+
- run: ./agent/scripts/error-if-diff.sh
35+
- run: ./gradlew compileKotlin
36+
working-directory: ./agent/src/main/kotlin

Diff for: .gitmodules

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
[submodule "agent/bindings/scip-typescript"]
2+
path = agent/bindings/scip-typescript
3+
url = https://github.com/sourcegraph/scip-typescript
4+
branch = olafurpg/signatures

Diff for: agent/bindings/kotlin/.gitattributes

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#
2+
# https://help.github.com/articles/dealing-with-line-endings/
3+
#
4+
# Linux start script should use lf
5+
/gradlew text eol=lf
6+
7+
# These are Windows script files and should use crlf
8+
*.bat text eol=crlf
9+

Diff for: agent/bindings/kotlin/.gitignore

+42
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm
2+
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
3+
4+
*.jar
5+
!gradle-wrapper.jar
6+
7+
# User local IDEA configuration files
8+
.idea/
9+
.run/
10+
11+
# IntelliJ project
12+
*.iml
13+
14+
# Build output & caches for IntelliJ plugin development
15+
build/
16+
idea-sandbox/
17+
.gradle/
18+
19+
## File-based project format:
20+
*.iws
21+
22+
# IntelliJ
23+
/out/
24+
25+
# mpeltonen/sbt-idea plugin
26+
.idea_modules/
27+
28+
# JIRA plugin
29+
atlassian-ide-plugin.xml
30+
31+
# Crashlytics plugin (for Android Studio and IntelliJ)
32+
com_crashlytics_export_strings.xml
33+
crashlytics.properties
34+
crashlytics-build.properties
35+
fabric.properties
36+
37+
# JavaScript resources
38+
src/main/resources/dist/*
39+
40+
# VSCode langserver artifacts
41+
/bin/
42+
.DS_Store

Diff for: agent/bindings/kotlin/build

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
/*
2+
* This file was generated by the Gradle 'init' task.
3+
*
4+
* This is a general purpose Gradle build.
5+
* To learn more about Gradle by exploring our Samples at https://docs.gradle.org/8.6/samples
6+
*/
7+
8+
dependencies {
9+
implementation("org.eclipse.lsp4j:org.eclipse.lsp4j.jsonrpc:0.21.0")
10+
}
11+
12+
java {
13+
toolchain {
14+
// Always compile the codebase with Java 11 regardless of what Java
15+
// version is installed on the computer. Gradle will download Java 11
16+
// even if you already have it installed on your computer.
17+
languageVersion.set(JavaLanguageVersion.of("8"))
18+
}
19+
}

Diff for: agent/bindings/kotlin/gradle/libs.versions.toml

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# This file was generated by the Gradle 'init' task.
2+
# https://docs.gradle.org/current/userguide/platforms.html#sub::toml-dependencies-format
3+
4+
[versions]
5+
commons-math3 = "3.6.1"
6+
guava = "32.1.3-jre"
7+
junit-jupiter-engine = "5.10.0"
8+
9+
[libraries]
10+
guava = { module = "com.google.guava:guava", version.ref = "guava" }
11+
junit-jupiter-engine = { module = "org.junit.jupiter:junit-jupiter-engine", version.ref = "junit-jupiter-engine" }
12+
13+
[plugins]
14+
jvm = { id = "org.jetbrains.kotlin.jvm", version = "1.9.20" }
42.4 KB
Binary file not shown.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
distributionBase=GRADLE_USER_HOME
2+
distributionPath=wrapper/dists
3+
distributionUrl=https\://services.gradle.org/distributions/gradle-8.6-bin.zip
4+
networkTimeout=10000
5+
validateDistributionUrl=true
6+
zipStoreBase=GRADLE_USER_HOME
7+
zipStorePath=wrapper/dists

Diff for: agent/bindings/kotlin/gradlew

+249
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,249 @@
1+
#!/bin/sh
2+
3+
#
4+
# Copyright © 2015-2021 the original authors.
5+
#
6+
# Licensed under the Apache License, Version 2.0 (the "License");
7+
# you may not use this file except in compliance with the License.
8+
# You may obtain a copy of the License at
9+
#
10+
# https://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
#
18+
19+
##############################################################################
20+
#
21+
# Gradle start up script for POSIX generated by Gradle.
22+
#
23+
# Important for running:
24+
#
25+
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
26+
# noncompliant, but you have some other compliant shell such as ksh or
27+
# bash, then to run this script, type that shell name before the whole
28+
# command line, like:
29+
#
30+
# ksh Gradle
31+
#
32+
# Busybox and similar reduced shells will NOT work, because this script
33+
# requires all of these POSIX shell features:
34+
# * functions;
35+
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
36+
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
37+
# * compound commands having a testable exit status, especially «case»;
38+
# * various built-in commands including «command», «set», and «ulimit».
39+
#
40+
# Important for patching:
41+
#
42+
# (2) This script targets any POSIX shell, so it avoids extensions provided
43+
# by Bash, Ksh, etc; in particular arrays are avoided.
44+
#
45+
# The "traditional" practice of packing multiple parameters into a
46+
# space-separated string is a well documented source of bugs and security
47+
# problems, so this is (mostly) avoided, by progressively accumulating
48+
# options in "$@", and eventually passing that to Java.
49+
#
50+
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
51+
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
52+
# see the in-line comments for details.
53+
#
54+
# There are tweaks for specific operating systems such as AIX, CygWin,
55+
# Darwin, MinGW, and NonStop.
56+
#
57+
# (3) This script is generated from the Groovy template
58+
# https://github.com/gradle/gradle/blob/HEAD/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
59+
# within the Gradle project.
60+
#
61+
# You can find Gradle at https://github.com/gradle/gradle/.
62+
#
63+
##############################################################################
64+
65+
# Attempt to set APP_HOME
66+
67+
# Resolve links: $0 may be a link
68+
app_path=$0
69+
70+
# Need this for daisy-chained symlinks.
71+
while
72+
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
73+
[ -h "$app_path" ]
74+
do
75+
ls=$( ls -ld "$app_path" )
76+
link=${ls#*' -> '}
77+
case $link in #(
78+
/*) app_path=$link ;; #(
79+
*) app_path=$APP_HOME$link ;;
80+
esac
81+
done
82+
83+
# This is normally unused
84+
# shellcheck disable=SC2034
85+
APP_BASE_NAME=${0##*/}
86+
# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
87+
APP_HOME=$( cd "${APP_HOME:-./}" > /dev/null && pwd -P ) || exit
88+
89+
# Use the maximum available, or set MAX_FD != -1 to use that value.
90+
MAX_FD=maximum
91+
92+
warn () {
93+
echo "$*"
94+
} >&2
95+
96+
die () {
97+
echo
98+
echo "$*"
99+
echo
100+
exit 1
101+
} >&2
102+
103+
# OS specific support (must be 'true' or 'false').
104+
cygwin=false
105+
msys=false
106+
darwin=false
107+
nonstop=false
108+
case "$( uname )" in #(
109+
CYGWIN* ) cygwin=true ;; #(
110+
Darwin* ) darwin=true ;; #(
111+
MSYS* | MINGW* ) msys=true ;; #(
112+
NONSTOP* ) nonstop=true ;;
113+
esac
114+
115+
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
116+
117+
118+
# Determine the Java command to use to start the JVM.
119+
if [ -n "$JAVA_HOME" ] ; then
120+
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
121+
# IBM's JDK on AIX uses strange locations for the executables
122+
JAVACMD=$JAVA_HOME/jre/sh/java
123+
else
124+
JAVACMD=$JAVA_HOME/bin/java
125+
fi
126+
if [ ! -x "$JAVACMD" ] ; then
127+
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
128+
129+
Please set the JAVA_HOME variable in your environment to match the
130+
location of your Java installation."
131+
fi
132+
else
133+
JAVACMD=java
134+
if ! command -v java >/dev/null 2>&1
135+
then
136+
die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
137+
138+
Please set the JAVA_HOME variable in your environment to match the
139+
location of your Java installation."
140+
fi
141+
fi
142+
143+
# Increase the maximum file descriptors if we can.
144+
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
145+
case $MAX_FD in #(
146+
max*)
147+
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
148+
# shellcheck disable=SC2039,SC3045
149+
MAX_FD=$( ulimit -H -n ) ||
150+
warn "Could not query maximum file descriptor limit"
151+
esac
152+
case $MAX_FD in #(
153+
'' | soft) :;; #(
154+
*)
155+
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
156+
# shellcheck disable=SC2039,SC3045
157+
ulimit -n "$MAX_FD" ||
158+
warn "Could not set maximum file descriptor limit to $MAX_FD"
159+
esac
160+
fi
161+
162+
# Collect all arguments for the java command, stacking in reverse order:
163+
# * args from the command line
164+
# * the main class name
165+
# * -classpath
166+
# * -D...appname settings
167+
# * --module-path (only if needed)
168+
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
169+
170+
# For Cygwin or MSYS, switch paths to Windows format before running java
171+
if "$cygwin" || "$msys" ; then
172+
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
173+
CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" )
174+
175+
JAVACMD=$( cygpath --unix "$JAVACMD" )
176+
177+
# Now convert the arguments - kludge to limit ourselves to /bin/sh
178+
for arg do
179+
if
180+
case $arg in #(
181+
-*) false ;; # don't mess with options #(
182+
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
183+
[ -e "$t" ] ;; #(
184+
*) false ;;
185+
esac
186+
then
187+
arg=$( cygpath --path --ignore --mixed "$arg" )
188+
fi
189+
# Roll the args list around exactly as many times as the number of
190+
# args, so each arg winds up back in the position where it started, but
191+
# possibly modified.
192+
#
193+
# NB: a `for` loop captures its iteration list before it begins, so
194+
# changing the positional parameters here affects neither the number of
195+
# iterations, nor the values presented in `arg`.
196+
shift # remove old arg
197+
set -- "$@" "$arg" # push replacement arg
198+
done
199+
fi
200+
201+
202+
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
203+
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
204+
205+
# Collect all arguments for the java command:
206+
# * DEFAULT_JVM_OPTS, JAVA_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
207+
# and any embedded shellness will be escaped.
208+
# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
209+
# treated as '${Hostname}' itself on the command line.
210+
211+
set -- \
212+
"-Dorg.gradle.appname=$APP_BASE_NAME" \
213+
-classpath "$CLASSPATH" \
214+
org.gradle.wrapper.GradleWrapperMain \
215+
"$@"
216+
217+
# Stop when "xargs" is not available.
218+
if ! command -v xargs >/dev/null 2>&1
219+
then
220+
die "xargs is not available"
221+
fi
222+
223+
# Use "xargs" to parse quoted args.
224+
#
225+
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
226+
#
227+
# In Bash we could simply go:
228+
#
229+
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
230+
# set -- "${ARGS[@]}" "$@"
231+
#
232+
# but POSIX shell has neither arrays nor command substitution, so instead we
233+
# post-process each arg (as a line of input to sed) to backslash-escape any
234+
# character that might be a shell metacharacter, then use eval to reverse
235+
# that process (while maintaining the separation between arguments), and wrap
236+
# the whole thing up as a single "set" statement.
237+
#
238+
# This will of course break if any of these variables contains a newline or
239+
# an unmatched quote.
240+
#
241+
242+
eval "set -- $(
243+
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
244+
xargs -n1 |
245+
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
246+
tr '\n' ' '
247+
)" '"$@"'
248+
249+
exec "$JAVACMD" "$@"

0 commit comments

Comments
 (0)