Merge pull request #11 from HaohanWang/dev

Release v0.0.1-beta

Author: PaulCCCCCCH

.dockerignore

@@ -0,0 +1,2 @@
+**/node_modules
+**/*.package-lock.json

.gitignore

@@ -1,8 +1,8 @@
-*/.DS_Store
-*/__pycache__
-*/.idea
-front-end/node_modules
-front-end/dist
+**/.DS_Store
+**/__pycache__
+**/.idea
+**/node_modules
+**/dist
 
 
 # local env files
@@ -15,6 +15,9 @@ yarn-debug.log*
 yarn-error.log*
 pnpm-debug.log*
 
+back-end/runs
+back-end/influence_list
+
 # Editor directories and files
 .idea
 .vscode
@@ -23,3 +26,4 @@ pnpm-debug.log*
 *.njsproj
 *.sln
 *.sw?
+*.pyc

Dockerfile

@@ -0,0 +1,30 @@
+FROM paulcccccch/robustar:base-0.0.1
+# Default VCUDA is 11.1
+ARG VCUDA=11.1 
+COPY . /Robustar2/
+
+WORKDIR /Robustar2
+
+
+## Host with lerna: Install global node dependencies
+# RUN npm install -g webpack webpack-cli lerna @vue/cli
+
+## Host with NGINX: 
+RUN sudo apt-get -y install nginx
+RUN rm -rf /var/www/html
+RUN cp -r /Robustar2/front-end/packages/robustar/dist /var/www/html
+
+# Install PyTorch 
+RUN ["/bin/bash", "-c", "./scripts/install_pytorch.sh -c $VCUDA"]
+
+# Install frontend libraries
+# WORKDIR /Robustar2/front-end
+# RUN npm install
+# RUN lerna bootstrap
+
+# Build image-editor
+# RUN lerna run build:editor
+# WORKDIR /Robustar2
+
+CMD ["/bin/bash", "./scripts/start.sh"]
+

README.md

@@ -1,5 +1,50 @@
 # Robustar_implementation
 
-#### Contributor
-- [Yuhao Zhang](mailto:yhao.zhang98@gmail.com)
-- [Leyang Hu](mailto:leonleyanghu@gmail.com)
+## To Run Docker
+
+First, run `robustar.sh -m setup -a <version_name>` to pull robustar image. 
+
+Then, run `robustar.sh -m run <options> `. For a list of `<options>`, please run `robustar.sh` with no arguments. Make sure you set up the mounting directories and port forwarding correctly.
+
+If at any point you wish to change the setting, please remove the docker container and setup a new one. You can run `docker container ls -a` to see a list of containers, and use `docker container rm <name>` to remove.
+
+Please make sure port 6848 and 6006 on your machine are available. 
+
+## Configuration File
+You need to pass a config file (default `./configs.json`) to `robustar.sh`. It is a `JSON` file with the following fields:
+
+- **weight_to_load**: The name of the weight file to be loaded. Robustar will display its predictions and attention weights on the given dataset. If not provided or file is not found, but `pre_trained` is set to true, Robustar will try to download a trained image somewhere else.
+- **model_arch**: The architecture of the model. Choose from `["resnet-18", "resnet-18-32x32", "resnet-18", "resnet-34", "resnet-50", "resnet-101", "resnet-152", "mobilenet-v2"]`. Make sure this matches what's stored in `weight_to_load`.
+]`
+- **device**: e.g. `'cpu'`, `'cuda'`, `'cuda:5'`, etc. Robustar uses this device to do both training and inference.
+- **pre_trained**: Do we load pre-trained weights? If set to false, `weight_to_load` will be ignored and Robustar will train a model from scratch. Note that the image predictions and focus will be non-sensical in this case.
+
+
+
+## Build Docker Image
+In front-end directory, run ` lerna run build `.   
+
+Then, return back to root directory and run
+```
+docker build --build-arg VCUDA=<cuda version> .
+```
+where `<cuda version>` is chosen from `cpu`, `9.2`, `10.2`, `11.1` and `11.3`.
+
+Adjust the tag of the docker image with
+```
+docker tag <image_id> <user_id>/<repo>:<version>
+```
+
+Finally, push onto DockerHub with:
+```
+docker push <user_id>/<repo>:<version>
+```
+
+## Dev setup 
+
+See [backend doc](./back-end/README.md) and [frontend doc](./front-end/README.md) for more details
+
+
+## Notes
+### Image URL
+For any dataset provided by the user, an `image_url` uniquely identifies an image. An `image_url` looks like `<split>/<image_id>`, i.e. it consists of a string `split` and ae natural number `image_id`, concatenated with a slash `/`. For example, `train/102` stands for the 102th image in the training set, and visiting `http://localhost:8000/image/train/102` gives you the image. Translation between `image_url` and its absolute path is performed at the backend.

back-end/.gitignore

@@ -0,0 +1,25 @@
+*/.DS_Store
+*/__pycache__
+*/.idea
+*/node_modules
+*/dist
+
+
+# local env files
+.env.local
+.env.*.local
+
+# Log files
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+pnpm-debug.log*
+
+# Editor directories and files
+.idea
+.vscode
+*.suo
+*.ntvs*
+*.njsproj
+*.sln
+*.sw?

back-end/Dockerfile

@@ -4,7 +4,7 @@
 ```
 cd back-end # If you are not already in this folder
 pip install -r requirements.txt
-python run server.py
+python server.py
 ```
 
 
@@ -21,11 +21,17 @@ https://hub.docker.com/r/cdonglin/robustar
 
 # Dev 
 
-Robustar reads from the following directories (absolute path, i.e. `Robustar2` folder is placed immediatly under `/` of your file system)
+Robustar reads from the following directories (absolute path, i.e. `Robustar2` folder is placed immediatly under `/` of your file system. Specifically, for linux and MacOS, put the folder under `/` directory. For Windows, put it directly under the volumn your operating system is installed (usually `C:`).
 
 - `/Robuster2/checkpoint_images` for pre-trained weights
-- `/Robuster2/dataset/train` for training dataset
+- `/Robuster2/dataset/train` for training dataset. Each subfolder under this directory should contain images for a class, i.e. it should follow the format of pytorch `ImageFolder`. Check [here](https://developpaper.com/detailed-explanation-of-the-use-of-imagefolder-in-pytorch/) for more details about `ImageFolder`
+- `/Robuster2/dataset/validation` for validation dataset
 - `/Robuster2/dataset/test` for test dataset
 - `/Robuster2/dataset/paired` for paired dataset
-- `/Robuster2/influence_images` for calculated influence 
+- `/Robuster2/influence_images` for calculated influence images
 - `/Robuster2/user-edit.json` for user edit.
+- `/Robuster2/configs.json` for server configs. You directly may copy `configs.json` in the repository over.
+
+You can download our example dataset [here](https://drive.google.com/file/d/1DTaPnWV91C9VXZ9JOBz7hoDxfyUXBpUv/view?usp=sharing)
+
+

back-end/README.md

@@ -0,0 +1,14 @@
+# import os
+# for module in os.listdir(os.path.dirname(__file__)):
+#     if module == '__init__.py' or module[-3:] != '.py':
+#         continue
+
+#     __import__("api.{}.*".format(module[:-3]), locals(), globals())
+# del module
+
+from .edit import *
+from .generate import *
+from .image import *
+from .predict import *
+from .train import *
+from .test import *

back-end/usage.py back-end/__init__.pyback-end/usage.py renamed to back-end/__init__.py

@@ -0,0 +1,41 @@
+from objects.RServer import RServer
+from objects.RResponse import RResponse
+from flask import request
+from io import BytesIO
+import numpy as np
+from PIL import Image
+import base64
+from utils.image_utils import imageURLToPath
+from utils.path_utils import get_paired_path
+
+server = RServer.getServer()
+app = server.getFlaskApp()
+dataManager = server.getDataManager()
+
+@app.route('/edit/<split>/<image_id>', methods=['POST'])
+def user_edit(split, image_id):
+    json_data = request.get_json()
+    encoded_string = json_data['image'].split(',')[1]
+    decoded = base64.b64decode(encoded_string)
+
+    h = int(json_data['image_height'])
+    w = int(json_data['image_width'])
+    with Image.open(BytesIO(decoded)) as img:
+        
+        img_path = imageURLToPath('{}/{}'.format(split, image_id))
+
+        # TODO: Maybe support editing other splits as well? Or not?
+        if split != 'train':
+            raise NotImplemented('Currently we only support editing the `train` split!')
+        paired_img_path = get_paired_path(img_path, dataManager.train_root, dataManager.paired_root)
+
+        to_save = img.resize((w, h))
+        to_save = to_save.convert('RGB') # image comming from canvas is RGBA
+
+        to_save.save(paired_img_path)
+
+    return RResponse.ok("Success!")
+
+
+if __name__ == '__main__':
+    print(RServer)