Query to geometry API by updating file input parameter

I’m trying to get output from a model where the only input parameter to the GH definition is a .txt file. It works all fine by using the Viewer API but I’m not sure how to query successfully the backend API.

I’m using this recomendation:

curl -X PUT https://sdr7euc1.eu-central-1.shapediver.com/api/v2/session/{sessionId}/output \
  --header 'Content-Type: application/json' \
  --data '{"PARAMETER_ID_1":"PARAMETER_VALUE_1","PARAMETER_ID_2":"PARAMETER_VALUE_2"}'

but I’m not sure what the PARAMETER_VALUE_1 should be if PARAMETER_ID_1 is a file input.

One of the idea was to parse my JSON object into a string but didn’t work.

What would be recommended procedure to send over the file as a parameter to the Geometry API?

Getting different error message now when trying to send only filename in the body and the file as form-data:

…so, once supplied with a URL for the input file, works fine :slight_smile:

ok, while reading the file as input from a URL brings up security issues on our side. The URL information must be public to work, which is not os good. Can you pleas suggest how to submit file value to a parameter?

Which language are you developing your backend application in? If it’s node.js I will create an example using our Geometry Backend SDK for JavaScript/TypeScript, which will make your life much easier.

Hi @snabela thanks a lot if you can make one example :slight_smile: …this part of our backend is in Python, but in anyways, if you make an example in NodeJS is perfectly fine, so we refactor it into Python :wink:

Just realized we already have an example for file upload using the SDK :wink:
Here it is. Please note that this example uses a file parameter which accepts images, not text files.

Summary of the steps you need to do:

  1. Request upload URL for a file-asset. Using this request you send the id of the file parameter for which you want to upload a file, the size of the file you want to upload, and its format (content type). Example:
{
  "e6a576af-2e16-43cc-a78f-d74ae1154205": {
    "format": "text/plain",
    "size": "1234"
  }
}

In case of success the response will contain an object asset.file where you can find the upload URL (href) and a new unique id of your uploaded file.

{
  "e6a576af-2e16-43cc-a78f-d74ae1154205": {
    "id": "id of your uploaded file",
    "href": "URL for uploading your file to"
  }
}
  1. Upload your file to the upload URL (href), using a PUT request.
  2. Send your customization request, using the id of your file (which you received in step 1) as the value for your file parameter.

thanks @snabela we will go through this :wink:

1 Like

@snabela thanks again for the workaround. However, we are still failing to manage all the way trough. All good until the point we upload the file to the provided URL. The file uploads but never really updates the geometry of our structure which supposed to be done by a new JSON or TXT file. This implementation works very ok on our front-end when we integrate with the Viewer API, V2 or even V3.
How can we make sure that our uploaded file is correct? We do not get any error values,

Probably you missed step 3 as explained above. After uploading the file, you need to set the file‘s id as value of the file parameter. (Note that this allows you to reuse files without uploading them again.) Please let me know whether that did the trick.

I’m sorry, I didn’t explain it well. Yes, I did that, after the file upload as suggested. I also get 200 OK, but the result is not reflecting my uploaded TXT file. It looks like, it fetches the default file parameters. In default I mean the file which is supplied to the GH definition before uploading it to SD.

Does it work when you test your model using the platform?

Yes, it does

Ok, in this case the problem is most likely not in the Grasshopper model but in the code using the viewer’s API. Could you create a minimal example which reproduces the problem please?

Hi Alexander, I’m Balazs’ colleague. I put together the following Python code for trying to figure this out. This is not runnable as-is, but maybe you could spot some misuse of your APIs anyway. Could there potentially be a bug in how you populate the “data”-tag from the /output-endpoint?

# prepare the input to grasshopper model
string_data = json.dumps(shapediver_input, default=datetime_to_string)

# prepare to start a new session
sd_ticket_backend = config.get(...)
sd_base_url = config.get(...)
sd_ticket_url = "/".join([sd_base_url, 'api/v2/ticket', sd_ticket_backend])

resp = requests.post(sd_ticket_url)
print("post to api/v2/ticket=", resp.status_code, "\n")

full_data = resp.json()
session_id = full_data["sessionId"]
params = full_data["parameters"]
for id, param in params.items():
	if param["name"] == "InputFromUI":
		input_param_id = id # get the ID of the InputFromUI input

print('session id=', session_id)
print("input_param_id=", input_param_id, "\n")

f_size = len(string_data)
print("size=", f_size)

# request upload url for file asset
sd_query_upload_url = "/".join([sd_base_url,'api/v2/session',session_id,'file/upload'])
headers = {'Content-Type': 'application/json'}
payload = json.dumps({input_param_id: {"format": "application/json", "size": f_size}})
upload_url_response = requests.post(sd_query_upload_url, headers=headers, data=payload)
print('post to api/v2/session/:id/file/upload=', upload_url_response.status_code, "\n")

f_upload_url = upload_url_response.json()["asset"]["file"][input_param_id]["href"]
file_id = upload_url_response.json()["asset"]["file"][input_param_id]["id"]
# print('upload url=', f_upload_url)
print('file_id=', file_id, "\n")

# perform upload to assigned URL
upload_response = requests.put(f_upload_url, json=string_data)
print('put to upload file=', upload_response.status_code, "\n")

# get output from model, using uploaded file as input
sd_output_url = "/".join([sd_base_url, 'api/v2/session/{}/output'.format(session_id)])
data = json.dumps({input_param_id: file_id})
headers = {'Content-type': 'application/json'}
print('customization params=', data, "\n")

sd_object = requests.put(sd_output_url, headers=headers, data=data)
print('put to api/v2/session/:id/output=', sd_object.status_code, "\n")

# extract relevant part of response
outputs = sd_object.json()["outputs"]
params = []
for key, param in outputs.items():
	if len(param["content"]) > 0:
		params.append(param["content"][0])

return params

This produces an output like this:

post to api/v2/ticket= 201

session id= 97a2926f-3a45-48f6-9ab0-850088becb5a
input_param_id= 77e1d03e-3e03-49f6-8d5f-ae5c122326d2

size= 43187
post to api/v2/session/:id/file/upload= 200

file_id= 4b58e006-e68f-49f3-b8e1-80d9f208a830

put to upload file= 200

customization params= {"77e1d03e-3e03-49f6-8d5f-ae5c122326d2": "4b58e006-e68f-49f3-b8e1-80d9f208a830"}

put to api/v2/session/:id/output= 200

As for the actual content returned from this code we would expect this (only pasting a small part of it):

    {
        "data": [
            1.0114815998326414,
            1.0114815998326414,
            0.8395958377695779,
            0.8395958377695779,
            0.6869420490842003,
            0.6869420490842003,
            0.4572,
            0.4572,
            0.4572,
            0.3556000000000001,
            0.3556000000000001,
            0.3556000000000001
        ],
        "name": "AreaOfMainLegsCircular",
        "plugin": "CommPlugin_1",
        "id": "16c1e4baa4fe2949a754a5950c066c16"
    },

…but we are receiving this:

    {
        "data": 2,
        "format": "data",
        "name": "AreaOfMainLegsCircular"
    },

Thank you for your help in this!
/Jonas

It would be good to see the complete response of the “ticket” request (POST), and the “output” request (PUT) for debugging this.

I would suggest that you specifically select the output you want to read content from, typically this is done by the name property of the output. Mixing content of different outputs might break your application in the future.

Hi Alexander, thank you for your suggestion!

I have attached two JSON files to this post which I think are the ones you are asking for. I was assuming you were interested in the response bodies only.

By the way, these outputs are not from the same run as the last post I made, so the file-id and session-id don’t match with my last post.

complete output of api-v2-session-id-output.json (21.6 KB)
complete output of api-v2-ticket-ticket.json (108.1 KB)

Alright, I was too optimistic, it won’t be possible to identify the problem without a minimal example. I suggest you do this:

  • Create a Grasshopper model that contains a single file input parameter, and a directly connected data output.
  • Upload this model to the platform, test it using the platform (you can use the browser console to see the data in the data output).
  • Test this model using your own code.

Ok, thanks a lot for your input!

This item might have to wait for a week or two now, because other tasks are more pressing at the moment.

We will be back! :slight_smile:

1 Like

Hi @snabela we have created a minimal example and followed your suggestion. Unfortunately the result is exactly the same, so no output.

This is the GH file: minimum-example.gh (11.8 KB)

Model name in SD: minimum_example you may find it under my account.

This is the example input file which we try to use it with the model: minimum_example_input.txt (31 Bytes)

And this is the result from the last call where we query the data at: https://sdeuc1.eu-central-1.shapediver.com/api/v2/session/:sessionId/output response:
response_minimum_example.json (1.6 KB)

To avoid coding errors, I have used Postman to follow along the steps and get response for each of them.

  • get session ID
  • get file upload URL
  • upload the file
  • get output

The get output step returns 200 but as you may see in the attached response JSON the data content is an empty array.

    "0eb3b6f3bc46851bfb7fa566d6fbaee3": {
      "id": "0eb3b6f3bc46851bfb7fa566d6fbaee3",
      "name": "SDDataOutput",
      "uid": "0d4be8e3-b313-414d-9841-855280c8596c",
      "version": "95f91126a079e41f7d277abe0ff4f0db",
      "order": 0,
      "tooltip": "",
      "displayname": "",
      "hidden": false,
      "status_computation": "success",
      "status_collect": "success",
      "content": []
    },

Please note that all works well if I use the SD UI to upload a new input file. Pls see below screenshot.

Ok, finally it has become clear what the problem is. I checked the files that were uploaded for the minimum_example model in your account. They were uploaded to the file upload url using Content-Type multipart/form-data. It’s interesting that this even is accepted by S3, because the Content-Type that you requested the pre-signed upload url for presumably is application/json. Probably S3 accepts this because the uploaded multipart data contains a single file of Content-Type application/json.
Anyway, the problem can easily be solved by uploading the plain file without packaging it into a multipart form request. Please give it a try.