SB5.1 - OCI - Ruby 2.x for Spark: Provision/DataPipeline/AnalyticsSpark

[heungheung]

I am trying to do a new installation. When running the Provision/DataPipeline/AnalyticsSpark job,

https://github.com/Sunbird-Obsrv/sunbird-data-pipeline/blob/release-5.0.0_RC1/ansible/roles/analytics-spark-provision/tasks/main.yml

Line 130-136

- name: Install latest ruby-dev
  become: yes
  apt:
    name: "ruby2.2-dev"
    state: installed
    update_cache: true
    cache_valid_time: 3600

In Jenkins Console

TASK [analytics-spark-provision : Install latest ruby-dev] *********************
fatal: [10.0.1.6]: FAILED! => {
    "changed": false
}

MSG:

No package matching 'ruby2.2-dev' is available

ssh to dp to check:

analytics@dp:~$ sudo apt list ruby2.*-dev
Listing... Done
ruby2.3-dev/bionic 2.3.8-4bbox1~bionic1 amd64
ruby2.4-dev/bionic 2.4.10-1bbox1~bionic1 amd64
ruby2.5-dev/bionic 2.5.8-1bbox1~bionic1 amd64
ruby2.6-dev/bionic 2.6.6-1bbox1~bionic1 amd64
ruby2.7-dev/bionic 2.7.5-1bbox1~bionic1 amd64

ubuntu@dp:~$ for X in /etc/apt/sources.list.d/*; do echo; echo; echo "** $X:"; echo; cat $X; done | grep ruby-ng
deb http://ppa.launchpad.net/brightbox/ruby-ng/ubuntu bionic main

root@dp:/etc# cat os-release 
NAME="Ubuntu"
VERSION="18.04.6 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.6 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

Looks like ruby2.2-dev is no longer available.

Is there any dependency for 2.2? Can we move to 2.3 or any later version?

Thanks!

[heungheung]

further deep dive
4.6 Doc: http://docs.sunbird.org/latest/developer-docs/server-installation/build-and-deploy/
Provision/DataPipeline/AnalyticsSpark is using tag release-4.6.0_RC3, role
https://github.com/Sunbird-Obsrv/sunbird-data-pipeline/blob/release-4.6.0_RC3/ansible/roles/analytics-spark-provision/tasks/main.yml
Line 120-136

- name: Install latest ruby
  become: yes
  become_user: "{{ analytics_user }}"
  shell: "export PATH=$PATH:/home/analytics/.rvm/bin && rvm install ruby-2.6"

- name: Add ruby repository
  become: yes
  apt_repository:
    repo: ppa:brightbox/ruby-ng

- name: Install latest ruby-dev
  become: yes
  apt:
    name: "ruby2.6-dev"
    state: installed
    update_cache: true
    cache_valid_time: 3600

were using Ruby 2.6

4.10 & 5.0 doc

• https://ed.sunbird.org/use/installing-ed-platform/release-4.10.0/build
• https://ed.sunbird.org/use/installing-ed-platform/release-5.0.0/build

Provision/DataPipeline/AnalyticsSpark are using tags release-4.10.5_RC3, release-5.0.0_RC1 respectively, both

• https://github.com/Sunbird-Obsrv/sunbird-data-pipeline/blob/release-4.10.5_RC3/ansible/roles/analytics-spark-provision/tasks/main.yml
• https://github.com/Sunbird-Obsrv/sunbird-data-pipeline/blob/release-5.0.0_RC1/ansible/roles/analytics-spark-provision/tasks/main.yml

are putting Ruby 2.2.

Ruby 2.2 does not have ruby2.2-dev for bionic.
http://ppa.launchpad.net/brightbox/ruby-ng/ubuntu/pool/main/r/ruby2.2/

I will assume Ruby should be 2.6 for 4.10 & 5.0. Please correct me if I am wrong

[SanthoshVasabhaktula]

@anandp504 - Can you check this once? We should get rid of the ruby dependency in the next release.

[heungheung]

looks like the same problem is in the 5.1.0 draft doc
https://ed.sunbird.org/use/installing-ed-platform/release-5.1.0-draft/build
is pointing the Provision/DataPipeline/AnalyticsSpark job to release-5.1.0 while
https://github.com/Sunbird-Obsrv/sunbird-data-pipeline/blob/release-5.1.0/ansible/roles/analytics-spark-provision/tasks/main.yml
is still using Ruby 2.2 in Line 120-141

[Krishnaj20]

@anandp504 - can you check and revert please?

[anandp504]

@heungheung Can you please try with ruby-2.5 and ruby2.5-dev packages? I tried it with 2.5 version and it works. There is no hard dependency on the ruby versions. We use a ruby script to push the data product configurations to kafka for scheduled execution. We will work on managing the ruby dependencies.

[heungheung]

Thanks for confirmation that there is no dependency with the Ruby version. We've already used 2.6 as it was being used in Release 4.6.0. I think there is no reason to go back for 1 minor Ruby version in 5.1.0

[heungheung]

Marking the discussion thread as answered - with using Ruby 2.6.