Shapediver "TextFileImport" restricted by Web Application Firewall - 406 error

Hi,
I have a problem with the Component: Text File Import and my host provider.
My host provider has a: Web Application Firewall - That block a lot of things.
I have contacted them, and their response was: “You need to tell the developers to set a specific User Agent”
The answer above is beyond my knowledge level.
My options are:
1 - I will have to disable the Firewall… = Not super nice
2 - Shapediver need to set a custom user agent. That way it will be allowed through the filter.
3 - Shapediver can provide me with the IP’s that you use, so I can create a custom rule to allow them through…
eg like this one:

<If “-R ‘35.171.0.122’”>
SecRuleRemoveByTag unoeuro

Br HC

I have also been trying to get the “importFile” components to work with Google drive, but with no luck.
Do you have a solution for Google drive? Because that would actually make it more easy than using my domain.
I seems like the problem is that Google Drive wants to use its build in “Viewer”. I can make it export directly using this URL: https://docs.google.com/uc?export=download&id=ID
But I can’t just “show” the file

Services like Dropbox or Google Drive provide different types of links, sometimes directly serving the requested file and sometimes providing a link to a hosted page where you can download the file. With dropbox, for example, one can switch between those two links by using the query ?dl=0 (hosted page) or ?dl=1 (direct download). Using this second option has been tested with ShapeDiver and it works without problems.

It seems that by default, Google Drive does not let you access the file itself. We have found this resource explaining how to get a direct link to the file. However, we did not test this solution yet. Let us know if you make it work.

Have created a simple txt file with direct download via Google drive:
https://docs.google.com/uc?export=download&id=1t0BvP1vuVbfnVxd0pgW6kHULbMmmWg6N

Can someone test if they can get it to load?
It should be the same as the dropbox direct download…

The link lets you download the file as expected. However I tried it with the file input component and it still doesn’t work. It seems that the MIME type of the file is not set to text/plain but to something else. I am not sure how much control Google gives you regarding these types, you might need to convert them before sending them to ShapeDiver.

@hcthisen many thanks for providing the link for testing. I did some deeper digging and the problem turns out to be this: Lots of hosting services provide default content types like application/binary, application/octet-stream, or text/plain. In those cases our file import component therefore guesses a content type based on the file extension. In your case that doesn’t work, because the filename is not exposed in the Google drive sharing link. Hence our component complains: image
We will somehow improve this behaviour in a future release.