Skip to content
This repository was archived by the owner on Dec 10, 2019. It is now read-only.

Support async exports #50

Merged
merged 15 commits into from
Mar 27, 2018
Merged

Support async exports #50

merged 15 commits into from
Mar 27, 2018

Conversation

jisantuc
Copy link
Contributor

@jisantuc jisantuc commented Mar 7, 2018

Overview

This PR adds support for asynchronous exports to analyses and projects. It includes:

  • an Export model to provide sugar around the bravado auto-generated API client
  • new or updated example notebooks for analyses and projects to show asynchronous exports
  • an example notebook for exports to show users everything they can do with Export objects
  • a change from get_export to get_thumbnail for synchronous exports
  • a create_export method for projects and analyses

Testing

  • run through the Export, Project, and Analysis example notebooks with an api pointed at
    staging

Supersedes #47

Closes #47

Closes #46

Closes #45

jisantuc and others added 10 commits February 28, 2018 16:27
Note that the export spec works for posting exports, but doesn't work for
listing exports. There's something wrong with telling bravado that `mask` is an
`object`, I think, or at least -- when you remove `mask` from the section on
exports `api.client.get_exports` works.
-Update models.
-Update spec.
-Update spec.
-Update gitignore.
@jisantuc
Copy link
Contributor Author

jisantuc commented Mar 12, 2018

I just spoke with @lewfish about what he needed to be able to do that spawned the issues behind this, and he needs two more things.

  • add files to export object on init
  • add raster size parameterization (in exportOptions)

-Update export model to expose configuration of rastersize on an export supported by backend.
-Update and clean up export notebook.
@jisantuc
Copy link
Contributor Author

@alkamin can you review this? I can't because I'm the PR author according to GitHub

Copy link
Contributor

@alkamin alkamin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also running into some swagger validation errors on the Analysis notebook.

"metadata": {},
"outputs": [],
"source": [
"project_export = project.create_export(bbox=bbox, zoom=8, raster_size=1000)"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Project.create_export doesn't take raster_size as an argument like Export.create_export does.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is resolved in the latest commit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now getting this error:

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
<ipython-input-5-c15771aff722> in <module>()
----> 1 project_export = project.create_export(bbox=bbox, zoom=8, raster_size=1000)

/Users/akaminsky/git/raster-foundry-python-client/rasterfoundry/models/project.py in create_export(self, bbox, zoom, raster_size)
    136         """
    137         return Export.create_export(
--> 138             self.api, bbox=bbox, zoom=zoom, project=self, raster_size=raster_size)
    139 
    140     def geotiff(self, bbox, zoom=10, raw=False):

/Users/akaminsky/git/raster-foundry-python-client/rasterfoundry/models/export.pyc in create_export(cls, api, bbox, zoom, project, analysis, visibility, source, export_type, raster_size)
    148         export_create.update(update_dict)
    149         return Export(
--> 150             api.client.Imagery.post_exports(Export=export_create).result(),
    151             api)
    152 

/Users/akaminsky/git/raster-foundry-python-client/venv/lib/python2.7/site-packages/bravado/client.pyc in __call__(self, **op_kwargs)
    241 
    242         request_params = construct_request(
--> 243             self.operation, request_options, **op_kwargs)
    244 
    245         http_client = self.operation.swagger_spec.http_client

/Users/akaminsky/git/raster-foundry-python-client/venv/lib/python2.7/site-packages/bravado/client.pyc in construct_request(operation, request_options, **op_kwargs)
    285             request[request_option] = request_options[request_option]
    286 
--> 287     construct_params(operation, request, op_kwargs)
    288 
    289     return request

/Users/akaminsky/git/raster-foundry-python-client/venv/lib/python2.7/site-packages/bravado/client.pyc in construct_params(operation, request, op_kwargs)
    308                 "{0} does not have parameter {1}"
    309                 .format(operation.operation_id, param_name))
--> 310         marshal_param(param, param_value, request)
    311 
    312     # Check required params and non-required params with a 'default' value

/Users/akaminsky/git/raster-foundry-python-client/venv/lib/python2.7/site-packages/bravado_core/param.pyc in marshal_param(param, value, request)
    122 
    123     if swagger_spec.config['validate_requests']:
--> 124         validate_schema_object(swagger_spec, param_spec, value)
    125 
    126     param_type = param_spec.get('type')

/Users/akaminsky/git/raster-foundry-python-client/venv/lib/python2.7/site-packages/bravado_core/validate.pyc in validate_schema_object(swagger_spec, schema_object_spec, value)
     57 
     58     elif is_object(swagger_spec, schema_object_spec):
---> 59         validate_object(swagger_spec, schema_object_spec, value)
     60 
     61     elif obj_type == 'file':

/Users/akaminsky/git/raster-foundry-python-client/venv/lib/python2.7/site-packages/bravado_core/validate.pyc in scrubbed(*args, **kwargs)
     31                 e.message = '*** ' + e.message[len(str(e.instance)):]
     32                 e.instance = '***'
---> 33             reraise(*sys.exc_info())
     34 
     35     return scrubbed

/Users/akaminsky/git/raster-foundry-python-client/venv/lib/python2.7/site-packages/bravado_core/validate.pyc in scrubbed(*args, **kwargs)
     23     def scrubbed(*args, **kwargs):
     24         try:
---> 25             return func(*args, **kwargs)
     26         except jsonschema.ValidationError as e:
     27             if (

/Users/akaminsky/git/raster-foundry-python-client/venv/lib/python2.7/site-packages/bravado_core/validate.pyc in validate_object(swagger_spec, object_spec, value)
    103         object_spec,
    104         format_checker=swagger_spec.format_checker,
--> 105         resolver=swagger_spec.resolver).validate(value)
    106 
    107 

/Users/akaminsky/git/raster-foundry-python-client/venv/lib/python2.7/site-packages/jsonschema/validators.pyc in validate(self, *args, **kwargs)
    128         def validate(self, *args, **kwargs):
    129             for error in self.iter_errors(*args, **kwargs):
--> 130                 raise error
    131 
    132         def is_type(self, instance, type):

ValidationError: 'TOBEEXPORTED' is not one of ['CREATED', 'EXPORTING', 'EXPORTED', 'QUEUED', 'PROCESSING', 'COMPLETE', 'FAILED', 'ABORTED']

Failed validating 'enum' in schema['allOf'][1]['properties']['exportStatus']:
    {'description': 'Status of export',
     'enum': ['CREATED',
              'EXPORTING',
              'EXPORTED',
              'QUEUED',
              'PROCESSING',
              'COMPLETE',
              'FAILED',
              'ABORTED'],
     'type': 'string'}

On instance['exportStatus']:
    'TOBEEXPORTED'

@aaronxsu aaronxsu force-pushed the feature/js/support-async-exports branch from 479e633 to ca91829 Compare March 22, 2018 02:45
@aaronxsu aaronxsu force-pushed the feature/js/support-async-exports branch from ca91829 to dc107b4 Compare March 22, 2018 02:53
@aaronxsu aaronxsu merged commit 9550e0a into develop Mar 27, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants