Code

Feature/hdfs#279

Code

Open

muthumala19 wants to merge 81 commits intomiyurud:mastermiyurud/jasminegraph:masterfrom muthumala19:feature/hdfs_conflict_resolvemuthumala19/jasminegraph:feature/hdfs_conflict_resolveCopy head branch name to clipboard

+2,149-130

Commits

Commits on Dec 29, 2024

• Implement fetching data from HDFS to partitions

  muthumala19

  committed
  5fab841

  Copy full SHA for 5fab841

• Add copyrights and license

  muthumala19

  committed
  eb0a48a

  Copy full SHA for eb0a48a

• Implement HashPartitioner class

  muthumala19

  committed
  e2aea11

  Copy full SHA for e2aea11

• Create HashPartitioner with separation of concern

  muthumala19

  committed
  bef3e04

  Copy full SHA for bef3e04

• Create HashPartitioner using thread per partition

  muthumala19

  committed
  83eb758

  Copy full SHA for 83eb758

• Temp: DataPublisher communication implementation

  muthumala19

  committed
  dc0187f

  Copy full SHA for dc0187f

• Temp: Implementation with DataPublisher objects

  muthumala19

  committed
  de981d2

  Copy full SHA for de981d2

• Create util method for master-worker communication

  muthumala19

  committed
  7f0659a

  Copy full SHA for 7f0659a

• Add code to distribute data to the partitions

  muthumala19

  committed
  5074141

  Copy full SHA for 5074141

• Change fetching,sending data from HDFS to workers

  muthumala19

  committed
  25daa9a

  Copy full SHA for 25daa9a

• Add validaton for HDFS dataset paths

  muthumala19

  committed
  ea4bf80

  Copy full SHA for ea4bf80

• Modify counting edges for HDFS implementation

  muthumala19

  committed
  b75d9fd

  Copy full SHA for b75d9fd

• Extend for handling undirected graphs

  muthumala19

  committed
  4212e46

  Copy full SHA for 4212e46

• Add copyrights and license

  muthumala19

  committed
  8150120

  Copy full SHA for 8150120

• Add updating uploadEndTime in metaDB

  muthumala19

  committed
  1a770e7

  Copy full SHA for 1a770e7

• Add integration tests for HDFS implementation

  muthumala19

  committed
  d7341b9

  Copy full SHA for d7341b9

• Merge branch 'miyurud:master' into feature/hdfs_dev

  muthumala19

  authored
  1c21ae5

  Copy full SHA for 1c21ae5

Commits on Dec 31, 2024

• Add recent prerequisites tag: 20241231T070657

  muthumala19

  committed
  6679c8b

  Copy full SHA for 6679c8b

• Fix cpplint issues

  muthumala19

  committed
  a8782c2

  Copy full SHA for a8782c2

• Fix cpplint issues in test.py

  muthumala19

  committed
  9833128

  Copy full SHA for 9833128

• Fix cpplint issues in test.py (ii)

  muthumala19

  committed
  915529e

  Copy full SHA for 915529e

Commits on Jan 2, 2025

• Fix invalid path to HDFS data in test-docker.sh

  muthumala19

  committed
  17deb1b

  Copy full SHA for 17deb1b

• Merge branch 'master' into feature/hdfs_dev

  muthumala19

  authored
  0ca9194

  Copy full SHA for 0ca9194

• Remove default HDFS server testing

  muthumala19

  committed
  273730f

  Copy full SHA for 273730f

• Merge remote-tracking branch 'origin/feature/hdfs_dev' into feature/hdfs_dev

  muthumala19

  committed
  93a08bc

  Copy full SHA for 93a08bc

• Set to the most recent prerequisites tag

  muthumala19

  committed
  de4cffa

  Copy full SHA for de4cffa

• Adjust for removal of default HDFS server testing

  muthumala19

  committed
  68e8885

  Copy full SHA for 68e8885

Commits on Jan 12, 2025

• Make variable name meaningful msg1->message1

  Show description for 43a1ee6
  

  muthumala19

  and

  miyurud

  committed
  43a1ee6

  Copy full SHA for 43a1ee6

• Replace with CARRIAGE_RETURN_NEW_LINE variable

  Show description for 164a212
  

  muthumala19

  and

  miyurud

  committed
  164a212

  Copy full SHA for 164a212

• Update error message

  Show description for 874869a
  

  muthumala19

  and

  miyurud

  committed
  874869a

  Copy full SHA for 874869a

• Update error message

  Show description for 90721cf
  

  muthumala19

  and

  miyurud

  committed
  90721cf

  Copy full SHA for 90721cf

• Replace with CARRIAGE_RETURN_NEW_LINE variable

  Show description for 44cb89a
  

  muthumala19

  and

  miyurud

  committed
  44cb89a

  Copy full SHA for 44cb89a

• Initialized "directed" variable as false

  Show description for 4071a38
  

  muthumala19

  and

  miyurud

  committed
  4071a38

  Copy full SHA for 4071a38

• Replace with CARRIAGE_RETURN_NEW_LINE variable

  Show description for 39b8452
  

  muthumala19

  and

  miyurud

  committed
  39b8452

  Copy full SHA for 39b8452

• Replace with CARRIAGE_RETURN_NEW_LINE variable

  Show description for 4cd0391
  

  muthumala19

  and

  miyurud

  committed
  4cd0391

  Copy full SHA for 4cd0391

• Replace with CARRIAGE_RETURN_NEW_LINE variable

  Show description for 8f3c893
  

  muthumala19

  and

  miyurud

  committed
  8f3c893

  Copy full SHA for 8f3c893

• Update TODO -> TODO(muthumala19)

  Show description for a52ec3b
  

  muthumala19

  and

  miyurud

  committed
  a52ec3b

  Copy full SHA for a52ec3b

• Update TODO -> TODO(muthumala19)

  Show description for f679c5d
  

  muthumala19

  and

  miyurud

  committed
  f679c5d

  Copy full SHA for f679c5d

• Correct mismatches after accepting suggestions

  muthumala19

  committed
  e9a9823

  Copy full SHA for e9a9823

• Update hdfs consumer.properties file

  muthumala19

  committed
  874799a

  Copy full SHA for 874799a

Commits on Jan 18, 2025

• Store hdfs file chunks in mounted location

  muthumala19

  committed
  3dafc47

  Copy full SHA for 3dafc47

Commits on Jan 27, 2025

• Update src/frontend/JasmineGraphFrontEnd.cpp

  miyurud

  authored
  0266261

  Copy full SHA for 0266261

• Correct spelling mistake

  muthumala19

  committed
  234db51

  Copy full SHA for 234db51

• Merge remote-tracking branch 'origin/feature/hdfs' into feature/hdfs

  muthumala19

  committed
  d9cfc74

  Copy full SHA for d9cfc74

Commits on Feb 4, 2025

• Add license to HDFSStreamHandler.cpp

  muthumala19

  committed
  112832d

  Copy full SHA for 112832d

• Simplify if statement according to the suggestion

  muthumala19

  committed
  d155e5a

  Copy full SHA for d155e5a

• Move hdfs configs into common config directory

  muthumala19

  committed
  ee222e2

  Copy full SHA for ee222e2

• Fix: graph status op->nop when session restart

  muthumala19

  committed
  7632658

  Copy full SHA for 7632658

• Create HDFS related deployable components for k8s

  muthumala19

  committed
  abc6396

  Copy full SHA for abc6396

• Rename containers

  muthumala19

  committed
  89455d8

  Copy full SHA for 89455d8

• Create PV,PVC for common configuration directory

  muthumala19

  committed
  54354af

  Copy full SHA for 54354af

• Add functionality to make hdfs pods ready

  muthumala19

  committed
  f0c1c2d

  Copy full SHA for f0c1c2d

• Refactor according to max length of a line

  muthumala19

  committed
  ef3b0fe

  Copy full SHA for ef3b0fe

• Enclose if statement with brackets

  muthumala19

  committed
  5a2f33c

  Copy full SHA for 5a2f33c

• Change header: JASMINEGRAPH_HASH_PARTITIONER_HEADER

  muthumala19

  committed
  bb1c4de

  Copy full SHA for bb1c4de

Commits on Feb 8, 2025

• Add prefix "hdfs" to hdfs deployable components

  muthumala19

  committed
  64209ff

  Copy full SHA for 64209ff

• Add cleaning hdfs resources

  muthumala19

  committed
  5f0ae09

  Copy full SHA for 5f0ae09

• Merge branch 'feature/hdfs' into feature/hdfs_conflict_resolve

  Show description for 962023a
  muthumala19

  committed
  962023a

  Copy full SHA for 962023a

• Resolve conflicts and change accordingly

  muthumala19

  committed
  b11be88

  Copy full SHA for b11be88

• Use isDirected property when adding edges

  muthumala19

  committed
  23a20d0

  Copy full SHA for 23a20d0

Commits on Feb 13, 2025

• Merge branch 'miyurud:master' into feature/hdfs_conflict_resolve

  muthumala19

  authored
  e6c6594

  Copy full SHA for e6c6594

• Remove -it: it gives error "Unable to use a TTY"

  muthumala19

  committed
  42eb966

  Copy full SHA for 42eb966

• Remove older implemented code

  muthumala19

  committed
  38516c5

  Copy full SHA for 38516c5

• Add memory limits to hdfs containers

  muthumala19

  committed
  f8880dd

  Copy full SHA for f8880dd

• Clean residual components before starting

  muthumala19

  committed
  41439e9

  Copy full SHA for 41439e9

• Fix shfmt issues

  muthumala19

  committed
  b45c6f6

  Copy full SHA for b45c6f6

• Fix shfmt issues (ii)

  muthumala19

  committed
  2acef27

  Copy full SHA for 2acef27

• Fix cpplint issues

  muthumala19

  committed
  e2db2b3

  Copy full SHA for e2db2b3

• Fix cpplint issues

  muthumala19

  committed
  a2e07b1

  Copy full SHA for a2e07b1

• Fix pylint issues

  muthumala19

  committed
  6495ac5

  Copy full SHA for 6495ac5

• Change message

  muthumala19

  committed
  33f2fb4

  Copy full SHA for 33f2fb4

• Add environment variables

  muthumala19

  committed
  ffcce9b

  Copy full SHA for ffcce9b

• Disable Datanode IP-hostname check in Namenode

  muthumala19

  committed
  7422fb3

  Copy full SHA for 7422fb3

• Change according to comments on the PR#274

  muthumala19

  committed
  9006759

  Copy full SHA for 9006759

Commits on Feb 27, 2025

• Add nodemanager and resource manager components

  muthumala19

  committed
  d5a056d

  Copy full SHA for d5a056d

Commits on Feb 28, 2025

• Fix pv and pvc binding issue

  muthumala19

  committed
  67f72bc

  Copy full SHA for 67f72bc

• Replace deprecated namenode port 50010->9870

  muthumala19

  committed
  c3b444c

  Copy full SHA for c3b444c

• Merge branch 'master' into feature/hdfs_conflict_resolve

  muthumala19

  authored
  ca2b6be

  Copy full SHA for ca2b6be

• Refactor SERVICE_PRECONDITION values

  muthumala19

  committed
  71001c6

  Copy full SHA for 71001c6

• Fix shfmt issues

  muthumala19

  committed
  d0f053a

  Copy full SHA for d0f053a

Commits on Mar 1, 2025

• Set DataNode hostname in StatefulSet config

  muthumala19

  committed
  215a3ed

  Copy full SHA for 215a3ed