Build and install microcks from scratch

[gabifericean]

Hello,
I am new on this topic and I am trying to build and install the microcks, but I encounter an error when I try to run a test using mickrocks cli.
I made next steps:

1. Clone https://github.com/microcks/microcks project

2. Install mongodb

3. Download microcks-cli

4. Build the web application using next commands:
   cd webapp
   mvn -Pprod package -DskipTests=true

5. Execute:
   export KEYCLOAK_ENABLED=false

6. Start webapp using next command:
   java -jar microcks-app-1.10.1-SNAPSHOT-exec.jar
   
   Microcks UI is accessible on http://localhost:8080/

7. Import async api definition using microcks-cli
   microcks-cli import docs/api/v1/public-api-spec.asyncapi.yaml:true --microcksURL=http://localhost:8080/api --keycloakClientId=dummy --keycloakClientSecret=dummy --insecure --verbose
   

8. Start a test using microcks-cli:
   microcks-cli test test-app:1.3.0 kafka://test-kafka-kafka-brokers.service.svc:9092/topic.1?startOffset=0 ASYNC_API_SCHEMA --microcksURL=http://localhost:8080/api/ --keycloakClientId=dummy --keycloakClientSecret=dummy --waitFor=60sec --insecure --verbose

At this step I receive next error:

• on microcks-cli: {"timestamp":1724081985844,"status":500,"error":"Internal Server Error","path":"/api/tests"}
• on microcks log: Cannot invoke "io.github.microcks.domain.Service.getId()" because "service" is null] with root cause
  

Do you have any idea what I missed for the setup?

Thank you!

[lbroudoux]

Hey @gabifericean

Thanks for trying this out! That's a good start but you're actually missing some ingredients 😉

As you're willing to work with Kafka and AsyncAPI, you need some more stuff:

1. In webapp/src/main/resources/config/application.properties, you need to set the async-api.enabled property to true
2. In webapp/src/main/resources/config/features.properties, you need to set the features.feature.async-api.enabled to true
3. As you're using the regular distro, you'll also need a Kafka broker. You can use the minions/async/redpanda.sh script to launch a local Kafka container that will be bound to localhost:9092 (the default in configuration files)
4. Then, you need to launch the Async Minion component located into the minions/async folder. You can simply run mvn quarkus:dev for that.

That way you'll have the required elements!

Then looking at your error in the log, I suspect that the service test-app:1.3.0 cannot be found. So first check that these are the correct name and version of the service that was discovered by importing the AsyncAPI file before. You can do this through the UI. Then, you may add quotes to prevent parsing errors: microcks-cli test 'test-app:1.3.0' kafka://localhost:9092/topic.1?startOffset=0 ASYNC_API_SCHEMA ...

It may be simpler for a first ride to do this via the UI.

[gabifericean]

@lbroudoux
Thank you for your response, was very useful.
The described setup is working fine, but now I need to connect to a Kafka topic using authentication (user/password). Do you know where I can specify credentials? I found this https://microcks.io/documentation/guides/administration/secrets/ in the documentation, but is not helping me for the current setup.
Thanks!

[lbroudoux]

Secrets must be used when you need authenticated connection to Kafka topics for tests.

To use authentication mechanisms when connecting to Kafka for mocking, you should specify those connection settings in the 2 different application.properties configuration files

1. For a template on what's possible to do on webapp/src/main/resources/config/application.properties, you may want to check this block in the Helm template,
2. For a template on what's possible to use on minions/async/src/main/resources/application.properties, you may want to check this block in the Helm template.

Those are basically the same capabilities but using Spring Boot or Quarkus configuration properties.

[gabifericean]

@lbroudoux I still struggling to configure microcks to work with Kafka that is configured to use OAUTHBEARER mechanism for authentication.

With the current setup, I tried to validate/test some messages on our Kafka topics (Kafka is part of our product). I don't use Kafka for mocking.
The use case is:

• import async definition (yaml file)
• execute test using microcks-cli. . Example of microcks-cli command: microcks-cli test {tag}:0.1.0 kafka://{my_brocker}:{my_port}/{my_topic}?startOffset=0 ASYNC_API_SCHEMA --microcksURL=http://localhost:8080/api --keycloakClientId=dummy --keycloakClientSecret=dummy --waitFor=60sec --insecure --verbose

Kafka is configured to use authentication. The problem that I have is to configure async minions to connect to the kafka using the SASL_SSL protocol.

I tried to use a jaas configuration file:
KafkaClient {
oauth.client.id={client_id}
oauth.client.secret={client_secret}
oauth.token.endpoint.uri={token_endpoint}
security.protocol=SASL_PLAINTEXT
sasl.mechanism=OAUTHBEARER
sasl.jaas.config='org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required ;'
sasl.login.callback.handler.class=io.strimzi.kafka.oauth.client.JaasClientOauthLoginCallbackHandler
};
and I changed next in the application.properties from async minions

Access to Kafka broker.
kafka.bootstrap.servers={my_broker}:{my_broker_port}
kafka.security.protocol=SASL_SSL
kafka.sasl.jaas.config="kafka_client_jaas.conf"

command to start Microcks: java -jar microcks-app-1.10.0-exec.jar
command to start async minions: mvn quarkus:dev

On async minions logs I see next error:

Any help will be highly appreciate it!
All setup is working fine if we are disable the authentication on Kafka!

Thank you for your support!

[lbroudoux]

OMG! I forgot to reply on this one... such a long delay: sorry for this.

kafka.bootstrap.servers={my_broker}:{my_broker_port}
kafka.security.protocol=SASL_SSL
kafka.sasl.jaas.config="kafka_client_jaas.conf"

The properties you've used below are actually only used when connecting to the Kafka broker used for mocking and not for testing. At the moment, there's no way to use OAUTHBEARER mechanism. We've just implement SSL at the moment but I admit it would be a terrific feature to have!