Questions tagged [jq]

jq is like sed for JSON data – you can use it to slice, filter, map, and transform structured data with the same ease that sed, awk, grep and friends let you play with text.

0
votes
1answer
15 views

Curl Response with JQ parising error : parse error: Invalid numeric literal at line 1, column 9

I ran into this issue and didn't find any reference on stackoverflow, someone may find this useful... Saving a json response of curl to a variable with -i flag may yield unexpected result. jq --...
-2
votes
1answer
30 views

how to pass multiple args into jq?

I am trying to write a new json file. I want to define multiple variables then set them with piping to different properties in the new json file. jq --arg dnb "$DOMAIN_NAME_BUILT" --arg origin "$...
0
votes
1answer
36 views

How to concatenate jq with variables?

I have a json object I am reading with jq and trying to write some properties with local variables. I am setting local variable in my shell script like so: LOCATION_NAME="stag5" DOMAIN_LOCATION="...
-1
votes
1answer
50 views

not able to convert json to csv

finding difficult to convert json to csv using jq command { "ResourceTagMappingList": [ { "ResourceARN": "arn:aws:ec2:us-east-1:027906322765228:customer-gateway/cgw-e404f4440d", "...
1
vote
1answer
27 views

Streaming without truncating

I have json data of the form below. I want to transform it, making the key of each record into a field of that record in a streaming fashion. My problem: I don't know how to do that without truncating ...
0
votes
0answers
38 views

How to find and replace json with shell variables using jq?

I have read properties with jq from a json object and have stored them to variables. I want to now read these variables and essentially find and replace a word inside the string with a global shell ...
0
votes
1answer
22 views

Why do adding jq arguments not change value [duplicate]

I'm writing a bash script and came across this problem. When I pass jq --arg flags, it is not updating the value. But when I hard code in the values without the --arg flags, it does. I have tried ...
0
votes
3answers
52 views

jq filtering sub-array values with main values of an object

I am having an issue figuring out what filters and methods I need to use in jq to transform my json data into what I am looking for. Every combination I have tried has ended up either not working at ...
0
votes
1answer
30 views

jq json tree traversal (npm node_modules)

The question concerns the code optimization. I want to define a json-file which specify what files will be copied and where. Namely, I want to apply it in npm-type project to transfer the files from ...
1
vote
2answers
95 views

Sort huge JSON file using bash or python

Requirement: I have a Json file which is in .gz format. So, when it is compressed it is around ~500 MB in size. When I extract it, the json file becomes nearly around ~10 GB. The extracted JSON file ...
0
votes
2answers
39 views

How to filter for key pairs in an object array (json and jq)

This is a follow-up on jq select error: "Cannot index string with string <object>" Previously, I can filter the entries in a json file that has the target objects with the following ...
0
votes
1answer
18 views

JQ: Bigint Numbers precision

I have actually a situation were i run into the BIGINT Problem and the haziness with that in jq 1.5/1.6 (in a Windows enviroment). I read the issue reports and thought that if i transform the number ...
-1
votes
1answer
37 views

Why is using $s in jq --argjson s “2000” different from using “2000” in the JQ code?

I am trying to write a bash script which parses from the nobel prize JSON file. The first argument is the year, and the second is the category. set -- 2000 physics # set $1 to 2000 and $2 to physics ...
1
vote
1answer
61 views

jq select error: “Cannot index string with string <object>”

command: cat test.json | jq -r '.[] | select(.input[] | .["$link"] | contains("randomtext1")) | .id' I was expecting to have both entries (a and b) to show up since they both contains randomtext1 ...
0
votes
1answer
23 views

jq select and contains error: null (null) and string (“randomtext1”) cannot have their containment checked

command: cat test.json | jq -r '.[] | select(.["$link"] | contains("randomtext1")).id' I was expecting to have both ids (a and b) show up as result of the above command since they both contains ...
0
votes
2answers
40 views

How to filter some array in a sub-object with object in a json file with jq

I need to filter a JSON with a nested strucutre like below. All objects in array b where attribute x contains a "z" in the value of x should be filtered out. The rest should stay in the file. { "...
1
vote
3answers
41 views

Filter json object if keys exists in another object

I have two files file1.json { "a": "x1", "b": "y1", "c": "z1" } file2.json { "a": "x2", "b": "y2" } Since a & b already exists in file2, I want to output a new object that contains ...
1
vote
2answers
28 views

Add a new field containing a list of values to an existing JSON using jq

I would like to add a variable IntendedFor with values scan1 and scan2 to an existing JSON file. I would like to do this with jq function, I tried: cat existing.json | jq '.IntendedFor |="["scan1", ...
-1
votes
0answers
19 views

join complex 2 JSON files based on common key with jq utility

I am merging the content of 2 websites in order to import the file content in 1 system and I managed to export the data from the sources in a json format. I am trying to merge the 2 exported json ...
5
votes
1answer
50 views

compare 2 json arrays and return the difference

We have a custom CD Pipeline Tool, which unfortunately does not version the deployment parameters. So I put these in a Bitbucket Repo as a json file and validate them against a REST API of this CD ...
0
votes
1answer
27 views

How can I process oneline json files using `jq`

I have a onelined json file that looks similar to this {"fieldA":1, "fieldB":"foo"} {"fieldA":2, "fieldB":"bar"} {"fieldA":4, "fieldB":"foobar"} ... How can I properly read this file using jq? I ...
4
votes
2answers
62 views

Transformation working on small files fails when used with the “--stream” option (required due to file size)

JQ play snippet: https://jqplay.org/s/D5-FZl8wOs I'm using jq to flatten a json array to be used for sql. json: { "0123":[ {"i":0,"p":"file 1","l":100}, {"i":1,"p":"file 2","l":200} ], ...
0
votes
1answer
34 views

Output based on a specific key (group_by)

I have the following JSON output. I would like the output to be grouped by the common TZ. Input: { "TZ": "5f105439-a0a7-4134-9e98-87553d434b35", "TN": "4c4c4544-0046-3310-8056-b2c04f433432", ...
1
vote
1answer
50 views

Filtering JSON by object name using jq

I'm unable to filter by keys using jq. # /home/test/show_param_db.sh -p memory -h host00* -f json | jq '.[]' > [ { > "host001": { > "status": "OK", > "msg": "", > ...
0
votes
2answers
44 views

How to filter a value using jq'

I am writing a script to create AWS volumes automatically. I am trying to filter out the value for volumeId using echo and jq [root@ip-172-41-23-8 ~]# aws ec2 describe-volumes --filters Name=status,...
0
votes
0answers
18 views

Store value for ECS Taskdef failures in variable and compare it under if condition

I am trying to script in groovy for jenkins pipeline job to capture any string for failures in taskdef in ECS and then based on that either update the service or recreate it. I'm using something like ...
0
votes
0answers
26 views

Jq Replace Existing JSON with Values Complex

I need to generate a CloudFormation parameter list (config.json) during a CodeBuild. I have a bash script in my repo that will take a couple of parameters. This will either be production, staging, or ...
0
votes
1answer
32 views

How to use shell script pipe json to curl to sent slack message [duplicate]

I have a monitor API (for example: www.example.com/monitor ) well return system status in JSON format, for example: { "foo" : 1000, "bar" : 100 } I want to use curl to get system status from ...
-1
votes
1answer
53 views

How to extract a value from JSON array?

I have the following json array. [ "bucketName=myBucket", ] It exists in a separate json file called parameters.json. It is the only thing in the file. I have a bash script that is trying to ...
-3
votes
1answer
44 views

JQ: Numeric field names

I use JQ 1.5 in a Windows10 enviroment (PowerShell). I built a jq statement that works on the example data on jqplay but throws a error on my enviroment: Sample: Code share Code: . | { last_update: ...
2
votes
1answer
71 views

Handle a very large input file without slurp

I am working with JSON output from a tool (massdns) that is formatted as follows: {"query_name":"1eaff.example.com.","query_type":"A","resp_name":"ns02.example.com.","resp_type":"A","data":"<ip>...
1
vote
2answers
59 views

Flatten nested dictionary to list of key:value pair

I am querying company house API for an anti-corruption investigation and I get results in a nested JSON which can be seen here. I would like to transform this dictionary into a simple {key:value, key:...
0
votes
1answer
48 views

JQ: Merging different objects into a new array

I use jq 1.5 in a Windows enviroment to modify json files. Hi have following json file: {"player_container":{"74016":["None","None","None","None","None","None","None","None"],"74002":["None","None","...
3
votes
2answers
36 views

Count grouped members of JSON array with jq 1.3 with inline output

I have a JSON file: [ { "instance_AZs": [ "us-east-1b", "us-east-1b", "us-east-1b", "us-east-1c", "us-east-1c", "us-east-1a", "us-east-1a"...
0
votes
2answers
83 views

Is it possible to have a dynamic selector in jq using shell variables?

I am curious if it is possible to have a shell variable like: SEARCH=".[] | select(.notes[] | select(.body|contains("[SOMETEXT]")))" and use that variable in a jq selection, like: jq '${SEARCH}' ...
-3
votes
1answer
36 views

modify nested object but maintain the full object after

I need to modify an element --an array-- (e.g.: "group-xyz") within a nested object in a JSON tree using JQ but once that's done then I need the entire object back with the modified data. The goal is ...
0
votes
1answer
99 views

jq how to get a return value based on condition

So I'm trying to figure out how to get the "CPU" value based on the condition from the value from "Tags"..."Value" So I want to print the "CPU" value if the value within the Tags is NO The return ...
0
votes
3answers
126 views

How to select an element with jq in a nested JSON

I have input like this: "data": [{ "id": 111585, "name": "Inverter", "batList": [{ "name": "Battery1", "dataDict": [{ "key": "b1_1", ...
0
votes
2answers
76 views

How to pipe each jq filter output to an external command?

I have a large file with a sequence of JSON objects. Each object has a value that I want to extract and run through an external program. How do you instruct jq to do this? For example, if I have ...
1
vote
2answers
86 views

Using jq to get a value, then using that value as a key to get another value

I have several hundred taxonomic IDs, and I am using curl and jq to extract relevant values from http://taxonomy.jgi-psf.org/tax/accession. Is there a way to pipe jq to itself so I can use the value ...
0
votes
2answers
51 views

Bash variable substitution for jq query always returns null

File export from company system generates json in a structure that I cannot change. When searching through for specific attributes based on variables within bash (v5.0.2), it always returns null when ...
1
vote
1answer
46 views

JQ applying function to dict but print surrounding datas

I'd like "1": { "id": "1", "type": "select", "label": "country :", "choices": { "1": { "label": "Canada CAN", "value": "", }...
-1
votes
1answer
15 views

Merge one object inside another object of json file

As I am working first time in shell scripts and json parser i.e. jq library. I wanted to merge details one of property of file1 and file2 inside another file3 along with rest of the property details. ...
0
votes
2answers
45 views

Building json path from JQ using some keyword

I have a deep json. Sometimes, I need to look for the json path for a key containing certain word. { "apiVersion": "v1", "kind": "Pod", "metadata": { "creationTimestamp": "2019-...
1
vote
2answers
36 views

jq cannot convert string to int in bash

I'm working on generating a new JSON payload to update Consul with a MSSQL database service location. When I call jq like this: mssql_svc_ip=$(kubectl get svc/mssql-linux -o 'jsonpath={.spec....
1
vote
2answers
55 views

jq - convert an array in a pair

I have a json array like this: ["abc", 0, "cd", 1] I'm struggling to convert this in a csv: "abc", 0 "cd", 1 It seems I'm far to find a solution, but I've tried with this: jq '[ "abc","def", "cd",...
0
votes
1answer
41 views

jq : mapping array in object to another object while keeping parent keys and adding new ones

I would like to map the following structure { "id" : "OUTER_ID", "name" : "OUTER_NAME" "items" : [ { "id" : "INNER_ID_1", "name" : "INNER_NAME_1", ...
-1
votes
0answers
38 views

How do you get initial-cluster required for adding a new member to etcd cluster from etcd?

When adding a new member to an etcd cluster, one needs to pass the flag --initial-cluster which accepts the members in a very a specific way, e.g. master-3-test=https://192.168.0.10:2379=master-1-...
0
votes
1answer
51 views

jq - Print leading and trailing context surrounding each match?

Is it possible to print the leading & trailing context of a match in jq? Say i have the following JSON: ... [{ "message": "Validating...", }, { "message": "Validated.", }, { "message": "...
0
votes
1answer
25 views

“jq: error: Cannot index array with string” getting data from ElasticSearch results with jq

I trying to get the listaObjeto from the json. This is an extract of my json: { "took": 42, "timed_out": false, "_shards": { "total": 5, "successful": 5, "skipped": 0, "failed":...

http://mssss.yulina-kosm.ru