Skip to content

Commit 011a7ff

Browse files
committed
getSparkSession() instead of spark variable
1 parent 2691253 commit 011a7ff

File tree

2 files changed

+4
-4
lines changed

2 files changed

+4
-4
lines changed

test/InMemPipeTestImages.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010

1111
from PIL import Image
1212

13-
df = (spark.read.json('/home/ubuntu/image_data/listings/metadata'))
13+
df = (getSparkSession().read.json('/home/ubuntu/image_data/listings/metadata'))
1414

1515
df = (
1616
df
@@ -31,7 +31,7 @@
3131
)
3232

3333
image_metadata = (
34-
spark
34+
getSparkSession()
3535
.read
3636
.csv(
3737
path='/home/ubuntu/image_data/images/metadata',

test/InMemPipeTestImagesJson.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010

1111
from PIL import Image
1212

13-
df = (spark.read.json('/home/ubuntu/image_data/listings/metadata'))
13+
df = (getSparkSession().read.json('/home/ubuntu/image_data/listings/metadata'))
1414

1515
df = (
1616
df
@@ -31,7 +31,7 @@
3131
)
3232

3333
image_metadata = (
34-
spark
34+
getSparkSession()
3535
.read
3636
.csv(
3737
path='/home/ubuntu/image_data/images/metadata',

0 commit comments

Comments
 (0)