Forum Discussion
Hi. I have a few things to share about this.
You don’t need to connect anything to the REST Get’s error view as a workaround. Just change the REST GET’s “Response entity type” setting from DEFAULT to BINARY. Follow that with a Mapper that maps $entity to $content (or whatever you want to call it), and use the Views section of the Mapper’s settings to change the output view’s Type from Document to Binary.
As for the File Reader’s failure to read the URL, there are a few interesting things going on here.
The File Reader is implemented a bit differently than the REST Get. The HTTP request it sends to the server has a User-Agent header whose value is the Java version you’re using. The HTTP requests sent by the REST Get snap don’t include a User-Agent header.
Oddly, the server responds differently depending on the value of the User-Agent header.
If you use Java 8, the User-Agent header is something like Java/1.8.0_181, and the server rejects the request with a 403 Forbidden response. I have no idea why the server is doing this. I’ve never seen this before.
If you use Java 11, the server responds with a 301 Moved Permanently, plus the header “Location: https://api.pluralsight.com:443/api-v0.9/courses”. This is much more normal. It’s known as a redirect.
The server is asking the client to use https instead of http to make the request. In other words, use SSL so the response will be encrypted. Many http clients will automatically follow such redirects by making another request. The REST Get snap automatically follows redirects when the “Follow redirects” setting is checked, which is why it works. Unfortunately, the File Reader does not follow redirects. If you use the File Reader with the http URL and you’re using Java 11, you’ll essentially get an HTML response with the “301 Moved Permanently” response, which isn’t very useful. However, to work around this, simply modify the File Reader’s File setting to specify https instead of http.
In short, the simplest way to read the CSV file is using a File Reader configured with the URL https://api.pluralsight.com/api-v0.9/courses, and make sure you’re using Java 11 for your plex.
- Aleksandar_A3 years agoContributor III
Hello @maahutch,
You can try by using the following expression in a Mapper Snap:
$.mapValues((v,k) =>v.mapValues((v1,k1) => typeof v1 == 'object' ? v1.subject_id.map(x => {"id":v.id,"subject_id":x}): false))
Then, you just split the array with JSON Splitter with the following JSON Path:
jsonPath($,"..subject")
Attached below is the sample pipeline.
–sl-comm-array-denormalized-table_2023_06_22.slp (5.1 KB)Please note that if you have another input structure this will not work.
Let me know if this helps you.
Best Regards,
Aleksandar. Thanks Aleksandar. Is there a resource to learn this notation and how to use it in Snaplogic?
If my JSON were formatted like this, how would I adapt that expression?
{ "id": "1", "subjects": [ { "subject_id":"a" }, { "subject_id":"b" } ] }
- Aleksandar_A3 years agoContributor III
For more resources and helpful insights please refer to the official SnapLogic documentation SnapLogic Documentation.
In order to format the new json you can use the following expression in a Mapper Snap:
$subjects.map(x => x.extend({"id":$id}))
Then use a JSON Splitter to split the subjects array.
You can also refer to the sample pipeline attached below:
–sl-comm-array-denormalized-table_2023_06_23.slp (9.0 KB)
Regards,
Aleksandar.