Wrong image data when fetching image

[vdkdamian]

New Issue Checklist

• I am not disclosing a vulnerability.
• I am not just asking a question.
• I have searched through existing issues.
• I can reproduce the issue with the latest version of Parse Swift.
• I can reproduce the issue with the latest version of Parse Server.

Issue Description

I am seeing issues when fetching an image. When i fetch an image for the first time, it loads the image corretly. But when it then tries to load for the second time, the image is broken. I looked at what the downloaded image looks like when opening in a text editor.

I noticed it doesn't contain data, but a path. This is an example:
file:\/\/\/var\/folders\/y5\/3qbps9t11cb2fz7lh1x8bw7w0000gn\/T\/com.app.myApp\/CFNetworkDownload_rDopeR.tmp

My uploaded images in the Database are valid images.

I want to point out that this way of fetching images always worked for me. It's since the Beta's that there is something wrong, but I don't find anything on the Apple Developer known issues page.

Steps to reproduce

This is an example of code I use.

let modelQuery =  Model.query()
modelQuery.find { result in
    switch result {
    case .success(let model):
        if let imageURL = model.image?.localURL {
            print("imageURL: \(imageURL)")
            if let imgData = try? Data(contentsOf: imageURL) {
                print("imgData: \(imgData)")
                print("imgData as string: \(String(decoding: imgData, as: UTF8.self))")
                self.imageData = imgData
            }
        } else {
            model.image?.fetch(completion: { result in
                switch result {
                case .success(let fetchedImage):
                    if let localImageUrl = fetchedImage.localURL {
                        print("localImageUrl: \(localImageUrl)")
                        if let imgData = try? Data(contentsOf: localImageUrl) {
                            print("imgData: \(imgData)")
                            print("imgData as string: \(String(decoding: imgData, as: UTF8.self))")
                            self.imageData = imgData
                        }
                    }
                case .failure(_):
                    break
                }
            })
        }
        self.model = model
    case .failure(_):
        break
    }
}

This is my log:

Actual Outcome

file:\/\/\/var\/folders\/y5\/3qbps9t11cb2fz7lh1x8bw7w0000gn\/T\/com.app.myApp\/CFNetworkDownload_rDopeR.tmp

Expected Outcome

Image data

Failing Test Case / Pull Request

• 🤩 I submitted a PR with a fix and a test case.
• 🧐 I submitted a PR with a failing test case.

Environment

Parse Swift

• SDK version: 1.9.8
• Operating system (iOS, macOS, watchOS, etc.): iOS, macOS
• Operating system version: iOS 15 Beta 8, Monterey 12.0 Beta 6

Server

• Parse Server version: 4.4.0
• Operating system: Back4App

Logs

localImageUrl: file:///Users/name/Library/Containers/com.app.myApp/Data/Library/Application%20Support/parse/com.app.myApp/Downloads/d48e0511e7660ab8e59fa2bb74cd48a4_1600.png
imgData: 123 bytes
imgData as string: "file:\/\/\/var\/folders\/y5\/3qbps9t11cb2fz7lh1x8bw7w0000gn\/T\/com.app.myApp\/CFNetworkDownload_rDopeR.tmp"

[cbaker6]

@novemTeam can you fill out the complete issue template? It looks like you removed information

[vdkdamian]

@cbaker6 I updated it. I still left out the Database info because I am just using Back4App

[cbaker6]

What happens when you use the latest version of Xcode 12? If this only occurs in Xcode 13 beta, it's probably not worth a fix until 13 goes Gold. If you can find some documentation that states Xcode 13 is changing the way downloaded data is handled, then I can take a look to see how to adjust.

[vdkdamian]

It did always work in Xcode 12.

[cbaker6]

PR #212 uses Xcode 13 beta 5. I believe there are test cases that cover this scenario that don't fail on Xcode 13. Can you look through them to verify?

Parse-Swift/Tests/ParseSwiftTests/ParseFileTests.swift

Lines 966 to 969 in 5994d63

	let fetchedFile2 = try parseFile.fetch(options: [.cachePolicy(.returnCacheDataDontLoad)])
	XCTAssertEqual(fetchedFile2.name, fetchedFile.name)
	XCTAssertEqual(fetchedFile2.url, fetchedFile.url)
	XCTAssertNotNil(fetchedFile2.localURL)

If I understand the issue you are saying is present,

Parse-Swift/Tests/ParseSwiftTests/ParseFileTests.swift

Line 969 in 5994d63

XCTAssertNotNil(fetchedFile2.localURL)

needs to check that there's an actual file at the URL?

Note that you can comment out these lines:

Parse-Swift/Tests/ParseSwiftTests/ParseFileTests.swift

Lines 944 to 955 in 5994d63

	let response = FileUploadResponse(name: "d3a37aed0672a024595b766f97133615_logo.svg",
	url: parseFileURL)
	let encoded: Data!
	do {
	encoded = try ParseCoding.jsonEncoder().encode(response)
	} catch {
	XCTFail("Should encode/decode. Error \(error)")
	return
	}
	MockURLProtocol.mockRequests { _ in
	return MockURLResponse(data: encoded, statusCode: 200, delay: 0.0)
	}

and set these to your Parse Server,

Parse-Swift/Tests/ParseSwiftTests/ParseFileTests.swift

Lines 31 to 35 in 5994d63

	ParseSwift.initialize(applicationId: "applicationId",
	clientKey: "clientKey",
	masterKey: "masterKey",
	serverURL: url,
	testing: true)

and test and debugger to see what's going on.

[vdkdamian]

@cbaker6 Sorry for the late response, kinda busy. I'll take a closer look this weekend.

[vdkdamian]

@cbaker6 I was unable to run the Tests because of lack of experience. But I do indeed think there should be a test to check if the file at .localURL is indeed a file.

Maybe something like this? I don't know if there is a better way to check if it is a image.

do {
     let data: Data = try Data(contentsOf: fetchedFile2.localURL!)
     XCTAssertGreaterThan(data.count, 200)
} catch {
     XCTFail("Should get data. Error \(error)")
     return
}

I also tried a few more things. I ran my project with Xcode 13 on my friends iPhone, and had the same issue. The images don't download correctly.

It also changes, 1/10 times it downloads correctly, the other times i get images with a path in it. See the screenshots

Here are some more screenshots

[cbaker6]

I checked this in Playgrounds (macOS) using Xcode 13 beta 5 (if you look in the folder directory, all of the ParseFiles are intact):

Parse-Swift/ParseSwift.playground/Pages/9 - Files.xcplaygroundpage/Contents.swift

Lines 42 to 97 in 1a30fb6

	//: Set the link online for the file.
	let linkToFile = URL(string: "https://parseplatform.org/img/logo.svg")!
	//: Create a new `ParseFile` for your picture.
	let profilePic = ParseFile(name: "profile.svg", cloudURL: linkToFile)
	//: Set the picture as part of your ParseObject
	score.profilePicture = profilePic
	/*: Save asynchronously (preferred way) - Performs work on background
	queue and returns to specified callbackQueue.
	If no callbackQueue is specified it returns to main queue.
	*/
	score.save { result in
	switch result {
	case .success(let savedScore):
	assert(savedScore.objectId != nil)
	assert(savedScore.createdAt != nil)
	assert(savedScore.updatedAt != nil)
	assert(savedScore.ACL == nil)
	assert(savedScore.score == 52)
	assert(savedScore.profilePicture != nil)
	print("Your profile picture has been successfully saved")
	//: To get the contents updated `ParseFile`, you need to fetch your GameScore.
	savedScore.fetch { result in
	switch result {
	case .success(let fetchedScore):
	guard let picture = fetchedScore.profilePicture,
	let url = fetchedScore.profilePicture?.url else {
	return
	}
	print("The new name of your saved profilePicture is: \(picture.name)")
	print("The profilePicture is saved to your Parse Server at: \(url)")
	print("The full details of your profilePicture ParseFile are: \(picture)")
	//: If you need to download your profilePicture
	picture.fetch { result in
	switch result {
	case .success(let fetchedFile):
	print("The file is now saved on your device at: \(String(describing: fetchedFile.localURL))")
	print("The full details of your profilePicture ParseFile are: \(fetchedFile)")
	case .failure(let error):
	assertionFailure("Error fetching: \(error)")
	}
	}
	case .failure(let error):
	assertionFailure("Error fetching: \(error)")
	}
	}
	case .failure(let error):
	assertionFailure("Error saving: \(error)")
	}
	}

as well as in a sample app (iOS) of mine that heavily uses ParseFile for pictures: https://github.com/netreconlab/SnapCat

I didn't see the issue you mentioned and everything seems to be working correctly. I recommend looking into other areas of how you are handling files in your app or maybe your server configuration.

Tested with Parse Server 4.5.0 and 4.10.3

[cbaker6]

You should also checkout your cache policy:

Parse-Swift/Sources/ParseSwift/Parse.swift

Lines 70 to 74 in 1a30fb6

	- parameter requestCachePolicy: The default caching policy for all http requests that determines
	when to return a response from the cache. Defaults to `useProtocolCachePolicy`. See Apple's [documentation](https://developer.apple.com/documentation/foundation/url_loading_system/accessing_cached_data)
	for more info.
	- parameter cacheMemoryCapacity: The memory capacity of the cache, in bytes. Defaults to 512KB.
	- parameter cacheDiskCapacity: The disk capacity of the cache, in bytes. Defaults to 10MB.

You can enable CORS in your index.js on your server by:

const cors = require('cors');

var app = express();
app.use(cors());
...

[vdkdamian]

@cbaker6 is this something I can change when using Back4App?

Also, did you run your test on iOS 15? Because I really can't see what I could be doing wrong when downloading images...

I just followed the documentation

[cbaker6]

@cbaker6 is this something I can change when using Back4App?

I’m not sure how Back4app works as I self-host, but if you have access to your index.js file you can add it.

Also, did you run your test on iOS 15? Because I really can't see what I could be doing wrong when downloading images...

My tests in #226 (comment) Used iOS 15 specifically and I didn’t have any issues.

[vdkdamian]

if let iconIcon = icon.iconImage {
    iconIcon.fetch { result in
        switch result {
        case .success(let fetchedFile):
            print("fetchedFile.data: \(String(describing: fetchedFile.data))")
            if let dataFromWebsiteIcon = try? Data(contentsOf: fetchedFile.localURL!) {
                print("dataFromWebsiteIcon: \(dataFromWebsiteIcon)")
                let websiteIcon = websiteFolder.appendingPathComponent("Icon.png")
                if fileManager.fileExists(atPath: websiteIcon.path) {
                    do {
                        try dataFromWebsiteIcon.write(to: websiteIcon)
                    } catch {
                        assertionFailure("Error writing Icon image: \(error)")
                    }
                } else {
                    fileManager.createFile(atPath: websiteIcon.path, contents: dataFromWebsiteIcon, attributes: nil)
                }
            } else {
                print("Error: couldn't get data from file.")
            }
        case .failure(let error):
            assertionFailure("Error fetching: \(error)")
        }
    }
}

I get these logs:

fetchedFile.data: nil
dataFromWebsiteIcon: 137 bytes

Really breaking my head around this. Can't find what's going on

[cbaker6]

This is how I fetch files:

https://github.com/netreconlab/SnapCat/blob/8218047ef311c7610da8f4cde6db0dd1108c0047/SnapCat/Utility.swift#L28-L44

[vdkdamian]

Update:
I changed my App Groups path name and then it downloaded my images again in a correct way. If I then delete the folder with the images, and redownload the icons, it's broken again.

So I am able to download images the first time, but then there is something wrong with the data at fetchedFile.localURL! if I read the data from this, I get that path, and not the image

[vdkdamian]

@cbaker6 Your code returns a UIImage, but I need the data.
I tried a part of your code:

if let path = image.localURL?.relativePath,
   let image = UIImage(contentsOfFile: path) {
    completion(image)
} else {
    completion(defaultImage)
}

then changed it to this :

if let path = image.localURL {
    if let dataImage = try? Data(contentsOf: path) {
        print("dataImage: \(dataImage)")
    } else {
        print("No Image")
    }
} else {
    print("No Image")
}

This also printed me that path, and not an image. (It printed 137 bytes, so I know this is that path line. It's way to small to be my image)

This makes me think that it has something to do with the fetch to the localURL, and not my code.. But it's weird you don't have this issue.

[vdkdamian]

Update:
I created a new project really quick, I have the same issue with this project

[cbaker6]

What happens when you don’t add “App Groups”?

[cbaker6]

I will not ba able to test out your zip files. Instead, you should attempt to make your case repeatable as a failing test case and submit the PR.

you also mentioned your server is 4.4.0, does the problem still persist on the latest server 4.10.3?

[vdkdamian]

These are the only ones I can set, I tried 4.2.0, 4.4.0, 4.5.0. None of them work. I don't know why you don't even bother try Testing my zip files... It was a project with all the keys set up, but OK.

For me the issue is repeatable. I tried everything.

[vdkdamian]

I went to the fetchedFile.localURL path on mac it clearly shows that the image is damaged when downloaded here, so no offense, but I don't think that has anything to do with my code.

Since I just read the contents of the file from this path and write the to my desired location, I think it must have something to do when downloading the image from the ParseSwift SDK.

The image below is in dutch, but it says that the image is damaged. As mentioned before, there is just a path in this file.
I tried 3 parse server versions, so I don't think the issue is from there neither.

[Xevro]

I tested the zip project on my computer and I had the same issue.
The first time the image was downloaded correctly, after deleting the folder it downloaded the image but it was damaged and could not be opened.

Xcode 13-beta-5
MacOS Monterey 12.0 bèta (21A5506j)

[cbaker6]

As I’ve stated previously #226 (comment), the zip files don’t mean much. It seems the issue may be something with “app groups” or even how you are using app groups and files. I don’t see this issue at all with the apps I pointed to using Xcode beta 5 or the RC. As I’ve also stated #226 (comment), you testing on a beta when the real version isn’t even out yet. Whatever the issue is, you need to create a failing testcase and a PR (as the issue template states) in order for it be addressed. If either of you submit a PR in the correct format, it will be reviewed.

[cbaker6]

Also, both of you are mentioning that you are not only testing on a beta Xcode, you are even testing on a beta Mac OS, Monterey. There’s not even CI for this in GitHub Actins. If you find a reasonable solution, add the correct test cases that don’t break the current CI, then I’ll review the PR.

[vdkdamian]

@cbaker6 iOS 15 launched yesterday, and Xcode is also out of Beta.

I found that it has nothing to do with App Groups. I will take a look at it next week. If I find a fix, I'll create a PR.

I also have this issue on both macOS as iOS.

[cbaker6]

@cbaker6 iOS 15 launched yesterday, and Xcode is also out of Beta.

I found that it has nothing to do with App Groups. I will take a look at it next week. If I find a fix, I'll create a PR.

I also have this issue on both macOS as iOS.

You tested on macOS Big Sur or Monterey? Monterey is still in beta

[cbaker6]

In addition, have you tried your apps on “real” iOS 15 devices. Using the released version of iOS 15, I don’t have any issues with posting and fetching photos in the apps I mentioned above.

I recommend checking and clarifying if you are 1) Running Monterey beta 2) running it on an Intel or M1 machine 3) Are your issues present on Big Sur 4) Do you have issues running on real iOS 15 devices?

I personally don’t develop on a beta macOS and I cannot replicate any of the issues mentioned on anything that is actually released, nor can the CI replicate it. If you choose to develop on beta software you are choosing to encounter the risk of buggy behavior and may need to modify code on your own to support.

[vdkdamian]

@cbaker6 Yes its on Monterey. Its running on an Intel iMac. But I actually started noticing the issue on the Beta software of iOS 15.

I'll know more next week to see if it still persists on the official version of iOS 15.

Also want to mention that this issue is not a really big problem for my app. I started recreating my app in Swift instead of objective c. So i then knew that ParseSwift was in a really early stage so things could fail.

I also decided to target iOS 15 and macOS Monterey since Swift and SwiftUI got some important/big improvements.

So the version I am developing now is not in production yet.

So next week I'll know more and investigate the issue more and keep you informed.

[vdkdamian]

@cbaker6 CFNetworkDownload_dqx5Vh.tmp: these files, are they created by Apple or by Parse? Cause I managed to locate them, and I saw that the first time downloading the image, this file looks like Screenshot 1, and then the second time, it look like Screenshot 2. So is it possible that this is an Apple issue?

Also want to point out, that I am now on the official version of iOS 15. But I still have this issue. First time fetching images is no issue. But fetching that exact same image a second time messes up the cache file I think.

To find these files I needed to dig in hidden folders, and found out that these are the folders where Apple caches things.
I think I need some help to find the real issue here.

[vdkdamian]

In the File API+Command, function downloadFile(_:) I changed a line of code in order to be able to inspect the cache files.
I changed it from
try FileManager.default.moveItem(at: tempFileLocation, to: fileLocation)
to
try FileManager.default.copyItem(at: tempFileLocation, to: fileLocation)

I made a video to highlight the problem, maybe this helps you understand a little bit better.

Dropbox video url

[cbaker6]

I updated one of my systems to Monterey and all of the ParseFileTests.swift passes. I also tried the SnapCat app I linked earlier and it worked fine.

You can test the ParseFileTests on your system, if it passes (which it should), then the problem is most likely something you are doing on your end and not the SDK.

[vdkdamian]

Hmm, I literately have no idea where I could be doing something wrong

[cbaker6]

I think you might be on to something. I have files that are uploaded with the ParseSwift SDK, and they work fine. The problem I am noticing is indeed with files I uploaded manually in the database.
But I upload every image as Icon.png. So the extension is defined in the name.

If you are uploading manually through the dashboard maybe the dashboard doesn't send the same info the Swift SDK sends when saving the photo. If so, it can possibly be a dashboard bug or, you are not uploading properly.

[vdkdamian]

If you are uploading manually through the dashboard maybe the dashboard doesn't send the same info the Swift SDK sends when saving the photo. If so, it can possibly be a dashboard bug or, you are not uploading properly.

Yes, maybe, but do you have an idea why the first download does work, but on second attempt not?

[cbaker6]

Yes, maybe, but do you have an idea why the first download does work, but on second attempt not?

I actually might... This may be a small bug related to caching. When you call your second fetch, do:

let fetchedFile2 = try parseFile.fetch(options: [.cachePolicy(.reloadIgnoringLocalCacheData)])

and tell me if it works. I'll post a potential patch soon for you to test.

[parse-github-assistant]

The label type:bug cannot be used in combination with type:question.

[vdkdamian]

I'll try out #272 for some of my users in a few days. But it seems like this fixed this issue

[cbaker6]

If a developer comes across this thread and is using Swift SDK < 2.2.0, the workaround was mentioned in #226 (comment) and #226 (comment). Essentially change the cache policy during your file fetch:

let fetchedFile2 = try parseFile.fetch(options: [.cachePolicy(.reloadIgnoringLocalCacheData)])

[vdkdamian]

@cbaker6 Is it correct that I need to add the (options: [.cachePolicy(.reloadIgnoringLocalCacheData)]) manually after my fetches?

I notice that unchanged file on the server return my corrupted data if I .fetch them for a second time.

Changing it to .fetch(options: [.cachePolicy(.reloadIgnoringLocalCacheData)]) fixes this, but I want to ask if it's correct that developers need to do this manually?

[cbaker6]

You would add .fetch(options: [.cachePolicy(.reloadIgnoringLocalCacheData)]) if you don't want to use the cached file, but in most cases, the default is to use the caching policy sent from the server containing the file. If your app cached the file before the bug fixes (< v2.2.0), you might want to delete the cache/app and see if it works after without adding the .reloadIgnoringLocalCacheData option

[vdkdamian]

If your app cached the file before the bug fixes (v2.2.1), you might want to delete the cache and see if it works after without adding the .reloadIgnoringLocalCacheData option

Then it's still a bug. I had a new file that got uploaded via ParseSwift, and on another device, the first download is fine. But on second attemp it's the corrupted filedata again.

Adding the custom cachePolicy fixes the issue for me. But I don't know if you think there might be a bug somewhere else?

[vdkdamian]

I think nobody has this issue because they are not fetching a static file multiple times. But in my case, I need it to be able to fetch a file multiple times.

It looks like if the file is already downloaded, it takes the path to that cached file and puts in the file Parse creates. But this only happens if the filename is the exact same, and if it already cached that file via .fetch.

I don't know i you think this is a issue, or worth spending time at. But for me it's fixed by adding the cachePolicy. Maybe for developers with the same issue?

[cbaker6]

Adding the custom cachePolicy fixes the issue for me. But I don't know if you think there might be a bug somewhere else?

I've tested it and don't see any issues. If you find a bug, you can open an issue. I recommend you look at my earlier comment #226 (comment) and if you can't find a bug or fix your problem, then add the cache policy .reloadIgnoringLocalCacheData option which will re-download the file every time you fetch instead of using cache.

[cbaker6]

It looks like if the file is already downloaded, it takes the path to that cached file and puts in the file Parse creates. But this only happens if the filename is the exact same, and if it already cached that file via .fetch.

This should work, fetching a file with the same url should yield the same file, so it's better to use cache. If you are changing the file, it should have a different url than the previous file or you should ignore cache.

[cbaker6]

Looking at your old screenshots in #226 (comment), the first file has some data, though I don’t know what it is, I will assume it’s correct. The second time you attemp to fetch the same file it should hit the cache and shouldn’t produce a .tmp file (unless you changed the url or filename). It should point to the parse directory containing the first file where it contains the correct file name and extension from the first download. The data contained in that file should be the exact same data from the first .tmp file.

[vdkdamian]

It should point to the parse directory containing the first file where it contains the correct file name and extension from the first download. The data contained in that file should be the exact same data from the first .tmp file.

That's not the case with me. It clearly show that it does something which results in the file data being a path instead of the actual data.

After work, I'll try to look where I can change the file from your #226 comment on back4app. Maybe that's the issue, I'll see...

[cbaker6]

I wouldn’t say it’s clear, at least not to me:

1. You haven’t shown updated screenshots with the latest version
2. You mentioned it only happens with photos you updated manually through the dashboard.
3. You mentioned it worked on previous OS’s
4. No one else has reported the issue. I use files extensively in a couple of my apps and fetch from cache for a better user experience and don’t see the issue.
5. You also seem to do something different with the data in file. During the fetch the file is located at the exact same location. You should make sure you are not accidentally overwriting the contents of the file in your code.

To me, you have a lot of moving parts that need to be debugged. If you use your app with another SDK that uses cache, that might provide some insight.

[cbaker6]

Be sure to clear you cache, ParseSwift.clearCache() when testing. The OS chooses to use the cache based on the request, meaning it may not choose to use cache. Just in case the OS choses not to cache you can check to see if the file is already downloaded and use it like I did in my repo: https://github.com/netreconlab/SnapCat/blob/f8fdcc0e039ea8ff62f9f0884804f8ee90575a0b/SnapCat/Utility.swift#L24-L60