YQ(1)

NAME
yq is a portable command-line data file processor

SYNOPSIS
yq [eval/eval-all] [expression] files..

eval/e - (default) Apply the expression to each document in each yaml file in sequence

eval-all/ea - Loads all yaml documents of all yaml files and runs expression once

DESCRIPTION
a lightweight and portable command-line data file processor. yq uses jq (https://github.com/stedolan/jq) like syntax but works with yaml, json, xml, csv, properties and TOML files. It doesn’t yet support everything jq does - but it does support the most common operations and functions, and more is being added continuously.

This documentation is also available at https://mikefarah.gitbook.io/yq/ # QUICK GUIDE

  Read a value:

        yq '.a.b[0].c' file.yaml

  Pipe from STDIN:

        cat file.yaml | yq '.a.b[0].c'

  Update a yaml file, in place

        yq -i '.a.b[0].c = "cool"' file.yaml

  Update using environment variables

        NAME=mike yq -i '.a.b[0].c = strenv(NAME)' file.yaml

  Merge multiple files

        yq ea '. as $item ireduce ({}; . * $item )' path/to/*.yml

Note the use of ea to evaluate all files at once (instead of in sequence.)

  Multiple updates to a yaml file

        yq -i '
          .a.b[0].c = "cool" |
          .x.y.z = "foobar" |
          .person.name = strenv(NAME)
        ' file.yaml

See the documentation (https://mikefarah.gitbook.io/yq/) for more.

KNOWN ISSUES / MISSING FEATURES

•   yq attempts to preserve comment positions and whitespace as much as possible, but it does not handle all scenarios (see https://github.com/go-yaml/yaml/tree/v3 for details)

•   Powershell has its own...opinions: https://mikefarah.gitbook.io/yq/usage/tips-and-tricks#quotes-in-windows-powershell

BUGS / ISSUES / FEATURE REQUESTS
Please visit the GitHub page https://github.com/mikefarah/yq/.

HOW IT WORKS
In yq expressions are made up of operators and pipes. A context of nodes is passed through the expression and each operation takes the context as input and returns a new context as output. That output is piped in as input for the next operation in the expression. To begin with, the context is set to the first yaml document of the first yaml file (if processing in sequence using eval).

Lets look at a couple of examples.

  Simple assignment example
Given a document like:

        a: cat
        b: dog

with an expression:

        .a = .b

Like math expressions - operator precedence is important.

The = operator takes two arguments, a lhs expression, which in this case is .a and rhs expression which is .b.

It pipes the current, lets call it `root' context through the lhs expression of .a to return the node

        cat

Side note: this node holds not only its value `cat', but comments and metadata too, including path and parent information.

The = operator then pipes the `root' context through the rhs expression of .b to return the node

        dog

Both sides have now been evaluated, so now the operator copies across the value from the RHS (.b) to the LHS (.a), and it returns the now updated context:

        a: dog
        b: dog

  Complex assignment, operator precedence rules
Just like math expressions - yq expressions have an order of precedence. The pipe | operator has a low order of precedence, so operators with higher precedence will get evaluated first.

Most of the time, this is intuitively what you’d want, for instance .a = "cat" | .b = "dog" is effectively: (.a = "cat") | (.b = "dog").

However, this is not always the case, particularly if you have a complex LHS or RHS expression, for instance if you want to select particular nodes to update.

Lets say you had:

        - name: bob
          fruit: apple
        - name: sally
          fruit: orange

Lets say you wanted to update the sally entry to have fruit: `mango'. The incorrect way to do that is: .[] | select(.name == "sally") | .fruit = "mango".

Because | has a low operator precedence, this will be evaluated (incorrectly) as : (.[]) | (select(.name == "sally")) | (.fruit = "mango"). What you’ll see is only the updated segment returned:

        name: sally
        fruit: mango

To properly update this yaml, you will need to use brackets (think BODMAS from maths) and wrap the entire LHS: (.[] | select(.name == "sally") | .fruit) = "mango"

Now that entire LHS expression is passed to the `assign' (=) operator, and the yaml is correctly updated and returned:

        - name: bob
          fruit: apple
        - name: sally
          fruit: mango

  Relative update (e.g. |=)
There is another form of the = operator which we call the relative form. It’s very similar to = but with one key difference when evaluating the RHS expression.

In the plain form, we pass in the `root' level context to the RHS expression. In relative form, we pass in each result of the LHS to the RHS expression. Let’s go through an example.

Given a document like:

        a: 1
        b: thing

with an expression:

        .a |= . + 1

Similar to the = operator, |= takes two operands, the LHS and RHS.

It pipes the current context (the whole document) through the LHS expression of .a to get the node value:

        1

Now it pipes that LHS context into the RHS expression . + 1 (whereas in the = plain form it piped the original document context into the RHS) to yield:

        2

The assignment operator then copies across the value from the RHS to the value on the LHS, and it returns the now updated `root' context:

        a: 2
        b: thing

Add
Add behaves differently according to the type of the LHS: * arrays: concatenate * number scalars: arithmetic addition * string scalars: concatenate * maps: shallow merge (use the multiply operator (*) to deeply merge)

Use += as a relative append assign for things like increment. Note that .a += .x is equivalent to running .a = .a + .x.

  Concatenate arrays
Given a sample.yml file of:

        a:
          - 1
          - 2
        b:
          - 3
          - 4

then

        yq '.a + .b' sample.yml

will output

        - 1
        - 2
        - 3
        - 4

  Concatenate to existing array
Note that the styling of a is kept.

Given a sample.yml file of:

        a: [1,2]
        b:
          - 3
          - 4

then

        yq '.a += .b' sample.yml

will output

        a: [1, 2, 3, 4]
        b:
          - 3
          - 4

  Concatenate null to array
Given a sample.yml file of:

        a:
          - 1
          - 2

then

        yq '.a + null' sample.yml

will output

        - 1
        - 2

  Append to existing array
Note that the styling is copied from existing array elements

Given a sample.yml file of:

        a: ['dog']

then

        yq '.a += "cat"' sample.yml

will output

        a: ['dog', 'cat']

  Prepend to existing array
Given a sample.yml file of:

        a:
          - dog

then

        yq '.a = ["cat"] + .a' sample.yml

will output

        a:
          - cat
          - dog

  Add new object to array
Given a sample.yml file of:

        a:
          - dog: woof

then

        yq '.a + {"cat": "meow"}' sample.yml

will output

        - dog: woof
        - cat: meow

  Relative append
Given a sample.yml file of:

        a:
          a1:
            b:
              - cat
          a2:
            b:
              - dog
          a3: {}

then

        yq '.a[].b += ["mouse"]' sample.yml

will output

        a:
          a1:
            b:
              - cat
              - mouse
          a2:
            b:
              - dog
              - mouse
          a3:
            b:
              - mouse

  String concatenation
Given a sample.yml file of:

        a: cat
        b: meow

then

        yq '.a += .b' sample.yml

will output

        a: catmeow
        b: meow

  Number addition - float
If the lhs or rhs are floats then the expression will be calculated with floats.

Given a sample.yml file of:

        a: 3
        b: 4.9

then

        yq '.a = .a + .b' sample.yml

will output

        a: 7.9
        b: 4.9

  Number addition - int
If both the lhs and rhs are ints then the expression will be calculated with ints.

Given a sample.yml file of:

        a: 3
        b: 4

then

        yq '.a = .a + .b' sample.yml

will output

        a: 7
        b: 4

  Increment numbers
Given a sample.yml file of:

        a: 3
        b: 5

then

        yq '.[] += 1' sample.yml

will output

        a: 4
        b: 6

  Date addition
You can add durations to dates. Assumes RFC3339 date time format, see date-time operators (https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information.

Given a sample.yml file of:

        a: 2021-01-01T00:00:00Z

then

        yq '.a += "3h10m"' sample.yml

will output

        a: 2021-01-01T03:10:00Z

  Date addition - custom format
You can add durations to dates. See date-time operators (https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information.

Given a sample.yml file of:

        a: Saturday, 15-Dec-01 at 2:59AM GMT

then

        yq 'with_dtf("Monday, 02-Jan-06 at 3:04PM MST", .a += "3h1m")' sample.yml

will output

        a: Saturday, 15-Dec-01 at 6:00AM GMT

  Add to null
Adding to null simply returns the rhs

Running

        yq --null-input 'null + "cat"'

will output

        cat

  Add maps to shallow merge
Adding objects together shallow merges them. Use * to deeply merge.

Given a sample.yml file of:

        a:
          thing:
            name: Astuff
            value: x
          a1: cool
        b:
          thing:
            name: Bstuff
            legs: 3
          b1: neat

then

        yq '.a += .b' sample.yml

will output

        a:
          thing:
            name: Bstuff
            legs: 3
          a1: cool
          b1: neat
        b:
          thing:
            name: Bstuff
            legs: 3
          b1: neat

  Custom types: that are really strings
When custom tags are encountered, yq will try to decode the underlying type.

Given a sample.yml file of:

        a: !horse cat
        b: !goat _meow

then

        yq '.a += .b' sample.yml

will output

        a: !horse cat_meow
        b: !goat _meow

  Custom types: that are really numbers
When custom tags are encountered, yq will try to decode the underlying type.

Given a sample.yml file of:

        a: !horse 1.2
        b: !goat 2.3

then

        yq '.a += .b' sample.yml

will output

        a: !horse 3.5
        b: !goat 2.3

Alternative (Default value)
This operator is used to provide alternative (or default) values when a particular expression is either null or false.

  LHS is defined
Given a sample.yml file of:

        a: bridge

then

        yq '.a // "hello"' sample.yml

will output

        bridge

  LHS is not defined
Given a sample.yml file of:

        {}

then

        yq '.a // "hello"' sample.yml

will output

        hello

  LHS is null
Given a sample.yml file of:

        a: ~

then

        yq '.a // "hello"' sample.yml

will output

        hello

  LHS is false
Given a sample.yml file of:

        a: false

then

        yq '.a // "hello"' sample.yml

will output

        hello

  RHS is an expression
Given a sample.yml file of:

        a: false
        b: cat

then

        yq '.a // .b' sample.yml

will output

        cat

  Update or create - entity exists
This initialises a if it’s not present

Given a sample.yml file of:

        a: 1

then

        yq '(.a // (.a = 0)) += 1' sample.yml

will output

        a: 2

  Update or create - entity does not exist
This initialises a if it’s not present

Given a sample.yml file of:

        b: camel

then

        yq '(.a // (.a = 0)) += 1' sample.yml

will output

        b: camel
        a: 1

Anchor and Alias Operators
Use the alias and anchor operators to read and write yaml aliases and anchors. The explode operator normalises a yaml file (dereference (or expands) aliases and remove anchor names).

yq supports merge aliases (like <<: *blah) however this is no longer in the standard yaml spec (1.2) and so yq will automatically add the !!merge tag to these nodes as it is effectively a custom tag.

  Merge one map
see https://yaml.org/type/merge.html

Given a sample.yml file of:

        - &CENTER
          x: 1
          y: 2
        - &LEFT
          x: 0
          y: 2
        - &BIG
          r: 10
        - &SMALL
          r: 1
        - !!merge <<: *CENTER
          r: 10

then

        yq '.[4] | explode(.)' sample.yml

will output

        x: 1
        y: 2
        r: 10

  Merge multiple maps
see https://yaml.org/type/merge.html

Given a sample.yml file of:

        - &CENTER
          x: 1
          y: 2
        - &LEFT
          x: 0
          y: 2
        - &BIG
          r: 10
        - &SMALL
          r: 1
        - !!merge <<:
            - *CENTER
            - *BIG

then

        yq '.[4] | explode(.)' sample.yml

will output

        r: 10
        x: 1
        y: 2

  Override
see https://yaml.org/type/merge.html

Given a sample.yml file of:

        - &CENTER
          x: 1
          y: 2
        - &LEFT
          x: 0
          y: 2
        - &BIG
          r: 10
        - &SMALL
          r: 1
        - !!merge <<:
            - *BIG
            - *LEFT
            - *SMALL
          x: 1

then

        yq '.[4] | explode(.)' sample.yml

will output

        r: 10
        x: 1
        y: 2

  Get anchor
Given a sample.yml file of:

        a: &billyBob cat

then

        yq '.a | anchor' sample.yml

will output

        billyBob

  Set anchor
Given a sample.yml file of:

        a: cat

then

        yq '.a anchor = "foobar"' sample.yml

will output

        a: &foobar cat

  Set anchor relatively using assign-update
Given a sample.yml file of:

        a:
          b: cat

then

        yq '.a anchor |= .b' sample.yml

will output

        a: &cat
          b: cat

  Get alias
Given a sample.yml file of:

        b: &billyBob meow
        a: *billyBob

then

        yq '.a | alias' sample.yml

will output

        billyBob

  Set alias
Given a sample.yml file of:

        b: &meow purr
        a: cat

then

        yq '.a alias = "meow"' sample.yml

will output

        b: &meow purr
        a: *meow

  Set alias to blank does nothing
Given a sample.yml file of:

        b: &meow purr
        a: cat

then

        yq '.a alias = ""' sample.yml

will output

        b: &meow purr
        a: cat

  Set alias relatively using assign-update
Given a sample.yml file of:

        b: &meow purr
        a:
          f: meow

then

        yq '.a alias |= .f' sample.yml

will output

        b: &meow purr
        a: *meow

  Explode alias and anchor
Given a sample.yml file of:

        f:
          a: &a cat
          b: *a

then

        yq 'explode(.f)' sample.yml

will output

        f:
          a: cat
          b: cat

  Explode with no aliases or anchors
Given a sample.yml file of:

        a: mike

then

        yq 'explode(.a)' sample.yml

will output

        a: mike

  Explode with alias keys
Given a sample.yml file of:

        f:
          a: &a cat
          *a: b

then

        yq 'explode(.f)' sample.yml

will output

        f:
          a: cat
          cat: b

  Explode with merge anchors
Given a sample.yml file of:

        foo: &foo
          a: foo_a
          thing: foo_thing
          c: foo_c
        bar: &bar
          b: bar_b
          thing: bar_thing
          c: bar_c
        foobarList:
          b: foobarList_b
          !!merge <<:
            - *foo
            - *bar
          c: foobarList_c
        foobar:
          c: foobar_c
          !!merge <<: *foo
          thing: foobar_thing

then

        yq 'explode(.)' sample.yml

will output

        foo:
          a: foo_a
          thing: foo_thing
          c: foo_c
        bar:
          b: bar_b
          thing: bar_thing
          c: bar_c
        foobarList:
          b: bar_b
          thing: foo_thing
          c: foobarList_c
          a: foo_a
        foobar:
          c: foo_c
          a: foo_a
          thing: foobar_thing

  Dereference and update a field
Use explode with multiply to dereference an object

Given a sample.yml file of:

        item_value: &item_value
          value: true
        thingOne:
          name: item_1
          !!merge <<: *item_value
        thingTwo:
          name: item_2
          !!merge <<: *item_value

then

        yq '.thingOne |= explode(.) * {"value": false}' sample.yml

will output

        item_value: &item_value
          value: true
        thingOne:
          name: item_1
          value: false
        thingTwo:
          name: item_2
          !!merge <<: *item_value

Array to Map
Use this operator to convert an array to..a map. The indices are used as map keys, null values in the array are skipped over.

Behind the scenes, this is implemented using reduce:

        (.[] | select(. != null) ) as $i ireduce({}; .[$i | key] = $i)

  Simple example
Given a sample.yml file of:

        cool:
          - null
          - null
          - hello

then

        yq '.cool |= array_to_map' sample.yml

will output

        cool:
          2: hello

Assign (Update)
This operator is used to update node values. It can be used in either the:

  plain form: =
Which will set the LHS node values equal to the RHS node values. The RHS expression is run against the matching nodes in the pipeline.

  relative form: |=
This will do a similar thing to the plain form, but the RHS expression is run with each LHS node as context. This is useful for updating values based on old values, e.g. increment.

  Flags

•   c clobber custom tags

  Create yaml file
Running

        yq --null-input '.a.b = "cat" | .x = "frog"'

will output

        a:
          b: cat
        x: frog

  Update node to be the child value
Given a sample.yml file of:

        a:
          b:
            g: foof

then

        yq '.a |= .b' sample.yml

will output

        a:
          g: foof

  Double elements in an array
Given a sample.yml file of:

        - 1
        - 2
        - 3

then

        yq '.[] |= . * 2' sample.yml

will output

        - 2
        - 4
        - 6

  Update node from another file
Note this will also work when the second file is a scalar (string/number)

Given a sample.yml file of:

        a: apples

And another sample another.yml file of:

        b: bob

then

        yq eval-all 'select(fileIndex==0).a = select(fileIndex==1) | select(fileIndex==0)' sample.yml another.yml

will output

        a:
          b: bob

  Update node to be the sibling value
Given a sample.yml file of:

        a:
          b: child
        b: sibling

then

        yq '.a = .b' sample.yml

will output

        a: sibling
        b: sibling

  Updated multiple paths
Given a sample.yml file of:

        a: fieldA
        b: fieldB
        c: fieldC

then

        yq '(.a, .c) = "potato"' sample.yml

will output

        a: potato
        b: fieldB
        c: potato

  Update string value
Given a sample.yml file of:

        a:
          b: apple

then

        yq '.a.b = "frog"' sample.yml

will output

        a:
          b: frog

  Update string value via |=
Note there is no difference between = and |= when the RHS is a scalar

Given a sample.yml file of:

        a:
          b: apple

then

        yq '.a.b |= "frog"' sample.yml

will output

        a:
          b: frog

  Update deeply selected results
Note that the LHS is wrapped in brackets! This is to ensure we don’t first filter out the yaml and then update the snippet.

Given a sample.yml file of:

        a:
          b: apple
          c: cactus

then

        yq '(.a[] | select(. == "apple")) = "frog"' sample.yml

will output

        a:
          b: frog
          c: cactus

  Update array values
Given a sample.yml file of:

        - candy
        - apple
        - sandy

then

        yq '(.[] | select(. == "*andy")) = "bogs"' sample.yml

will output

        - bogs
        - apple
        - bogs

  Update empty object
Given a sample.yml file of:

        {}

then

        yq '.a.b |= "bogs"' sample.yml

will output

        a:
          b: bogs

  Update node value that has an anchor
Anchor will remain

Given a sample.yml file of:

        a: &cool cat

then

        yq '.a = "dog"' sample.yml

will output

        a: &cool dog

  Update empty object and array
Given a sample.yml file of:

        {}

then

        yq '.a.b.[0] |= "bogs"' sample.yml

will output

        a:
          b:
            - bogs

  Custom types are maintained by default
Given a sample.yml file of:

        a: !cat meow
        b: !dog woof

then

        yq '.a = .b' sample.yml

will output

        a: !cat woof
        b: !dog woof

  Custom types: clobber
Use the c option to clobber custom tags

Given a sample.yml file of:

        a: !cat meow
        b: !dog woof

then

        yq '.a =c .b' sample.yml

will output

        a: !dog woof
        b: !dog woof

Boolean Operators
The or and and operators take two parameters and return a boolean result.

not flips a boolean from true to false, or vice versa.

any will return true if there are any true values in an array sequence, and all will return true if all elements in an array are true.

any_c(condition) and all_c(condition) are like any and all but they take a condition expression that is used against each element to determine if it’s true. Note: in jq you can simply pass a condition to any or all and it simply works - yq isn’t that clever..yet

These are most commonly used with the select operator to filter particular nodes.

  Related Operators

•   equals / not equals (==, !=) operators here (https://mikefarah.gitbook.io/yq/operators/equals)

•   comparison (>=, < etc) operators here (https://mikefarah.gitbook.io/yq/operators/compare)

•   select operator here (https://mikefarah.gitbook.io/yq/operators/select)

  or example
Running

        yq --null-input 'true or false'

will output

        true

  “yes” and “no” are strings
In the yaml 1.2 standard, support for yes/no as booleans was dropped - they are now considered strings. See `10.2.1.2. Boolean' in https://yaml.org/spec/1.2.2/

Given a sample.yml file of:

        - yes
        - no

then

        yq '.[] | tag' sample.yml

will output

        !!str
        !!str

  and example
Running

        yq --null-input 'true and false'

will output

        false

  Matching nodes with select, equals and or
Given a sample.yml file of:

        - a: bird
          b: dog
        - a: frog
          b: bird
        - a: cat
          b: fly

then

        yq '[.[] | select(.a == "cat" or .b == "dog")]' sample.yml

will output

        - a: bird
          b: dog
        - a: cat
          b: fly

  any returns true if any boolean in a given array is true
Given a sample.yml file of:

        - false
        - true

then

        yq 'any' sample.yml

will output

        true

  any returns false for an empty array
Given a sample.yml file of:

        []

then

        yq 'any' sample.yml

will output

        false

  any_c returns true if any element in the array is true for the given condition.
Given a sample.yml file of:

        a:
          - rad
          - awesome
        b:
          - meh
          - whatever

then

        yq '.[] |= any_c(. == "awesome")' sample.yml

will output

        a: true
        b: false

  all returns true if all booleans in a given array are true
Given a sample.yml file of:

        - true
        - true

then

        yq 'all' sample.yml

will output

        true

  all returns true for an empty array
Given a sample.yml file of:

        []

then

        yq 'all' sample.yml

will output

        true

  all_c returns true if all elements in the array are true for the given condition.
Given a sample.yml file of:

        a:
          - rad
          - awesome
        b:
          - meh
          - 12

then

        yq '.[] |= all_c(tag == "!!str")' sample.yml

will output

        a: true
        b: false

  Not true is false
Running

        yq --null-input 'true | not'

will output

        false

  Not false is true
Running

        yq --null-input 'false | not'

will output

        true

  String values considered to be true
Running

        yq --null-input '"cat" | not'

will output

        false

  Empty string value considered to be true
Running

        yq --null-input '"" | not'

will output

        false

  Numbers are considered to be true
Running

        yq --null-input '1 | not'

will output

        false

  Zero is considered to be true
Running

        yq --null-input '0 | not'

will output

        false

  Null is considered to be false
Running

        yq --null-input '~ | not'

will output

        true

Collect into Array
This creates an array using the expression between the square brackets.

  Collect empty
Running

        yq --null-input '[]'

will output

        []

  Collect single
Running

        yq --null-input '["cat"]'

will output

        - cat

  Collect many
Given a sample.yml file of:

        a: cat
        b: dog

then

        yq '[.a, .b]' sample.yml

will output

        - cat
        - dog

Column
Returns the column of the matching node. Starts from 1, 0 indicates there was no column data.

Column is the number of characters that precede that node on the line it starts.

  Returns column of value node
Given a sample.yml file of:

        a: cat
        b: bob

then

        yq '.b | column' sample.yml

will output

        4

  Returns column of key node
Pipe through the key operator to get the column of the key

Given a sample.yml file of:

        a: cat
        b: bob

then

        yq '.b | key | column' sample.yml

will output

        1

  First column is 1
Given a sample.yml file of:

        a: cat

then

        yq '.a | key | column' sample.yml

will output

        1

  No column data is 0
Running

        yq --null-input '{"a": "new entry"} | column'

will output

        0

Comment Operators
Use these comment operators to set or retrieve comments. Note that line comments on maps/arrays are actually set on the key node as opposed to the value (map/array). See below for examples.

Like the = and |= assign operators, the same syntax applies when updating comments:

  plain form: =
This will set the LHS nodes’ comments equal to the expression on the RHS. The RHS is run against the matching nodes in the pipeline

  relative form: |=
This is similar to the plain form, but it evaluates the RHS with each matching LHS node as context. This is useful if you want to set the comments as a relative expression of the node, for instance its value or path.

  Set line comment
Set the comment on the key node for more reliability (see below).

Given a sample.yml file of:

        a: cat

then

        yq '.a line_comment="single"' sample.yml

will output

        a: cat # single

  Set line comment of a maps/arrays
For maps and arrays, you need to set the line comment on the key node. This will also work for scalars.

Given a sample.yml file of:

        a:
          b: things

then

        yq '(.a | key) line_comment="single"' sample.yml

will output

        a: # single
          b: things

  Use update assign to perform relative updates
Given a sample.yml file of:

        a: cat
        b: dog

then

        yq '.. line_comment |= .' sample.yml

will output

        a: cat # cat
        b: dog # dog

  Where is the comment - map key example
The underlying yaml parser can assign comments in a document to surprising nodes. Use an expression like this to find where you comment is. `p' indicates the path, `isKey' is if the node is a map key (as opposed to a map value). From this, you can see the `hello-world-comment' is actually on the `hello' key

Given a sample.yml file of:

        hello: # hello-world-comment
          message: world

then

        yq '[... | {"p": path | join("."), "isKey": is_key, "hc": headComment, "lc": lineComment, "fc": footComment}]' sample.yml

will output

        - p: ""
          isKey: false
          hc: ""
          lc: ""
          fc: ""
        - p: hello
          isKey: true
          hc: ""
          lc: hello-world-comment
          fc: ""
        - p: hello
          isKey: false
          hc: ""
          lc: ""
          fc: ""
        - p: hello.message
          isKey: true
          hc: ""
          lc: ""
          fc: ""
        - p: hello.message
          isKey: false
          hc: ""
          lc: ""
          fc: ""

  Retrieve comment - map key example
From the previous example, we know that the comment is on the `hello' key as a lineComment

Given a sample.yml file of:

        hello: # hello-world-comment
          message: world

then

        yq '.hello | key | line_comment' sample.yml

will output

        hello-world-comment

  Where is the comment - array example
The underlying yaml parser can assign comments in a document to surprising nodes. Use an expression like this to find where you comment is. `p' indicates the path, `isKey' is if the node is a map key (as opposed to a map value). From this, you can see the `under-name-comment' is actually on the first child

Given a sample.yml file of:

        name:
          # under-name-comment
          - first-array-child

then

        yq '[... | {"p": path | join("."), "isKey": is_key, "hc": headComment, "lc": lineComment, "fc": footComment}]' sample.yml

will output

        - p: ""
          isKey: false
          hc: ""
          lc: ""
          fc: ""
        - p: name
          isKey: true
          hc: ""
          lc: ""
          fc: ""
        - p: name
          isKey: false
          hc: ""
          lc: ""
          fc: ""
        - p: name.0
          isKey: false
          hc: under-name-comment
          lc: ""
          fc: ""

  Retrieve comment - array example
From the previous example, we know that the comment is on the first child as a headComment

Given a sample.yml file of:

        name:
          # under-name-comment
          - first-array-child

then

        yq '.name[0] | headComment' sample.yml

will output

        under-name-comment

  Set head comment
Given a sample.yml file of:

        a: cat

then

        yq '. head_comment="single"' sample.yml

will output

        # single
        a: cat

  Set head comment of a map entry
Given a sample.yml file of:

        f: foo
        a:
          b: cat

then

        yq '(.a | key) head_comment="single"' sample.yml

will output

        f: foo
        # single
        a:
          b: cat

  Set foot comment, using an expression
Given a sample.yml file of:

        a: cat

then

        yq '. foot_comment=.a' sample.yml

will output

        a: cat
        # cat

  Remove comment
Given a sample.yml file of:

        a: cat # comment
        b: dog # leave this

then

        yq '.a line_comment=""' sample.yml

will output

        a: cat
        b: dog # leave this

  Remove (strip) all comments
Note the use of ... to ensure key nodes are included.

Given a sample.yml file of:

        # hi
        a: cat # comment
        # great
        b: # key comment

then

        yq '... comments=""' sample.yml

will output

        a: cat
        b:

  Get line comment
Given a sample.yml file of:

        # welcome!
        a: cat # meow
        # have a great day

then

        yq '.a | line_comment' sample.yml

will output

        meow

  Get head comment
Given a sample.yml file of:

        # welcome!
        a: cat # meow
        # have a great day

then

        yq '. | head_comment' sample.yml

will output

        welcome!

  Head comment with document split
Given a sample.yml file of:

        # welcome!
        ---
        # bob
        a: cat # meow
        # have a great day

then

        yq 'head_comment' sample.yml

will output

        welcome!
        bob

  Get foot comment
Given a sample.yml file of:

        # welcome!
        a: cat # meow
        # have a great day
        # no really

then

        yq '. | foot_comment' sample.yml

will output

        have a great day
        no really

Compare Operators
Comparison operators (>, >=, <, <=) can be used for comparing scalar values of the same time.

The following types are currently supported:

•   numbers

•   strings

•   datetimes

  Related Operators

•   equals / not equals (==, !=) operators here (https://mikefarah.gitbook.io/yq/operators/equals)

•   boolean operators (and, or, any etc) here (https://mikefarah.gitbook.io/yq/operators/boolean-operators)

•   select operator here (https://mikefarah.gitbook.io/yq/operators/select)

  Compare numbers (>)
Given a sample.yml file of:

        a: 5
        b: 4

then

        yq '.a > .b' sample.yml

will output

        true

  Compare equal numbers (>=)
Given a sample.yml file of:

        a: 5
        b: 5

then

        yq '.a >= .b' sample.yml

will output

        true

  Compare strings
Compares strings by their bytecode.

Given a sample.yml file of:

        a: zoo
        b: apple

then

        yq '.a > .b' sample.yml

will output

        true

  Compare date times
You can compare date times. Assumes RFC3339 date time format, see date-time operators (https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information.

Given a sample.yml file of:

        a: 2021-01-01T03:10:00Z
        b: 2020-01-01T03:10:00Z

then

        yq '.a > .b' sample.yml

will output

        true

  Both sides are null: > is false
Running

        yq --null-input '.a > .b'

will output

        false

  Both sides are null: >= is true
Running

        yq --null-input '.a >= .b'

will output

        true

Contains
This returns true if the context contains the passed in parameter, and false otherwise. For arrays, this will return true if the passed in array is contained within the array. For strings, it will return true if the string is a substring.

{% hint style=“warning” %}

Note that, just like jq, when checking if an array of strings contains another, this will use contains and not equals to check each string. This means an expression like contains(["cat"]) will return true for an array ["cats"].

See the “Array has a subset array” example below on how to check for a subset.

{% endhint %}

  Array contains array
Array is equal or subset of

Given a sample.yml file of:

        - foobar
        - foobaz
        - blarp

then

        yq 'contains(["baz", "bar"])' sample.yml

will output

        true

  Array has a subset array
Subtract the superset array from the subset, if there’s anything left, it’s not a subset

Given a sample.yml file of:

        - foobar
        - foobaz
        - blarp

then

        yq '["baz", "bar"] - . | length == 0' sample.yml

will output

        false

  Object included in array
Given a sample.yml file of:

        "foo": 12
        "bar":
          - 1
          - 2
          - "barp": 12
            "blip": 13

then

        yq 'contains({"bar": [{"barp": 12}]})' sample.yml

will output

        true

  Object not included in array
Given a sample.yml file of:

        "foo": 12
        "bar":
          - 1
          - 2
          - "barp": 12
            "blip": 13

then

        yq 'contains({"foo": 12, "bar": [{"barp": 15}]})' sample.yml

will output

        false

  String contains substring
Given a sample.yml file of:

        foobar

then

        yq 'contains("bar")' sample.yml

will output

        true

  String equals string
Given a sample.yml file of:

        meow

then

        yq 'contains("meow")' sample.yml

will output

        true

Create, Collect into Object
This is used to construct objects (or maps). This can be used against existing yaml, or to create fresh yaml documents.

  Collect empty object
Running

        yq --null-input '{}'

will output

        {}

  Wrap (prefix) existing object
Given a sample.yml file of:

        name: Mike

then

        yq '{"wrap": .}' sample.yml

will output

        wrap:
          name: Mike

  Using splat to create multiple objects
Given a sample.yml file of:

        name: Mike
        pets:
          - cat
          - dog

then

        yq '{.name: .pets.[]}' sample.yml

will output

        Mike: cat
        Mike: dog

  Working with multiple documents
Given a sample.yml file of:

        name: Mike
        pets:
          - cat
          - dog
        ---
        name: Rosey
        pets:
          - monkey
          - sheep

then

        yq '{.name: .pets.[]}' sample.yml

will output

        Mike: cat
        Mike: dog
        ---
        Rosey: monkey
        Rosey: sheep

  Creating yaml from scratch
Running

        yq --null-input '{"wrap": "frog"}'

will output

        wrap: frog

  Creating yaml from scratch with multiple objects
Running

        yq --null-input '(.a.b = "foo") | (.d.e = "bar")'

will output

        a:
          b: foo
        d:
          e: bar

Date Time
Various operators for parsing and manipulating dates.

  Date time formattings
This uses Golang’s built in time library for parsing and formatting date times.

When not specified, the RFC3339 standard is assumed 2006-01-02T15:04:05Z07:00 for parsing.

To specify a custom parsing format, use the with_dtf operator. The first parameter sets the datetime parsing format for the expression in the second parameter. The expression can be any valid yq expression tree.

        yq 'with_dtf("myformat"; .a + "3h" | tz("Australia/Melbourne"))'

See the library docs (https://pkg.go.dev/time#pkg-constants) for examples of formatting options.

  Timezones
This uses Golang’s built in LoadLocation function to parse timezones strings. See the library docs (https://pkg.go.dev/time#LoadLocation) for more details.

  Durations
Durations are parsed using Golang’s built in ParseDuration (https://pkg.go.dev/time#ParseDuration) function.

You can add durations to time using the + operator.

  Format: from standard RFC3339 format
Providing a single parameter assumes a standard RFC3339 datetime format. If the target format is not a valid yaml datetime format, the result will be a string tagged node.

Given a sample.yml file of:

        a: 2001-12-15T02:59:43.1Z

then

        yq '.a |= format_datetime("Monday, 02-Jan-06 at 3:04PM")' sample.yml

will output

        a: Saturday, 15-Dec-01 at 2:59AM

  Format: from custom date time
Use with_dtf to set a custom datetime format for parsing.

Given a sample.yml file of:

        a: Saturday, 15-Dec-01 at 2:59AM

then

        yq '.a |= with_dtf("Monday, 02-Jan-06 at 3:04PM"; format_datetime("2006-01-02"))' sample.yml

will output

        a: 2001-12-15

  Format: get the day of the week
Given a sample.yml file of:

        a: 2001-12-15

then

        yq '.a | format_datetime("Monday")' sample.yml

will output

        Saturday

  Now
Given a sample.yml file of:

        a: cool

then

        yq '.updated = now' sample.yml

will output

        a: cool
        updated: 2021-05-19T01:02:03Z

  From Unix
Converts from unix time. Note, you don’t have to pipe through the tz operator :)

Running

        yq --null-input '1675301929 | from_unix | tz("UTC")'

will output

        2023-02-02T01:38:49Z

  To Unix
Converts to unix time

Running

        yq --null-input 'now | to_unix'

will output

        1621386123

  Timezone: from standard RFC3339 format
Returns a new datetime in the specified timezone. Specify standard IANA Time Zone format or `utc', `local'. When given a single parameter, this assumes the datetime is in RFC3339 format.

Given a sample.yml file of:

        a: cool

then

        yq '.updated = (now | tz("Australia/Sydney"))' sample.yml

will output

        a: cool
        updated: 2021-05-19T11:02:03+10:00

  Timezone: with custom format
Specify standard IANA Time Zone format or `utc', `local'

Given a sample.yml file of:

        a: Saturday, 15-Dec-01 at 2:59AM GMT

then

        yq '.a |= with_dtf("Monday, 02-Jan-06 at 3:04PM MST"; tz("Australia/Sydney"))' sample.yml

will output

        a: Saturday, 15-Dec-01 at 1:59PM AEDT

  Add and tz custom format
Specify standard IANA Time Zone format or `utc', `local'

Given a sample.yml file of:

        a: Saturday, 15-Dec-01 at 2:59AM GMT

then

        yq '.a |= with_dtf("Monday, 02-Jan-06 at 3:04PM MST"; tz("Australia/Sydney"))' sample.yml

will output

        a: Saturday, 15-Dec-01 at 1:59PM AEDT

  Date addition
Given a sample.yml file of:

        a: 2021-01-01T00:00:00Z

then

        yq '.a += "3h10m"' sample.yml

will output

        a: 2021-01-01T03:10:00Z

  Date subtraction
You can subtract durations from dates. Assumes RFC3339 date time format, see date-time operators (https://mikefarah.gitbook.io/yq/operators/datetime#date-time-formattings) for more information.

Given a sample.yml file of:

        a: 2021-01-01T03:10:00Z

then

        yq '.a -= "3h10m"' sample.yml

will output

        a: 2021-01-01T00:00:00Z

  Date addition - custom format
Given a sample.yml file of:

        a: Saturday, 15-Dec-01 at 2:59AM GMT

then

        yq 'with_dtf("Monday, 02-Jan-06 at 3:04PM MST"; .a += "3h1m")' sample.yml

will output

        a: Saturday, 15-Dec-01 at 6:00AM GMT

  Date script with custom format
You can embed full expressions in with_dtf if needed.

Given a sample.yml file of:

        a: Saturday, 15-Dec-01 at 2:59AM GMT

then

        yq 'with_dtf("Monday, 02-Jan-06 at 3:04PM MST"; .a = (.a + "3h1m" | tz("Australia/Perth")))' sample.yml

will output

        a: Saturday, 15-Dec-01 at 2:00PM AWST

Delete
Deletes matching entries in maps or arrays.

  Delete entry in map
Given a sample.yml file of:

        a: cat
        b: dog

then

        yq 'del(.b)' sample.yml

will output

        a: cat

  Delete nested entry in map
Given a sample.yml file of:

        a:
          a1: fred
          a2: frood

then

        yq 'del(.a.a1)' sample.yml

will output

        a:
          a2: frood

  Delete entry in array
Given a sample.yml file of:

        - 1
        - 2
        - 3

then

        yq 'del(.[1])' sample.yml

will output

        - 1
        - 3

  Delete nested entry in array
Given a sample.yml file of:

        - a: cat
          b: dog

then

        yq 'del(.[0].a)' sample.yml

will output

        - b: dog

  Delete no matches
Given a sample.yml file of:

        a: cat
        b: dog

then

        yq 'del(.c)' sample.yml

will output

        a: cat
        b: dog

  Delete matching entries
Given a sample.yml file of:

        a: cat
        b: dog
        c: bat

then

        yq 'del( .[] | select(. == "*at") )' sample.yml

will output

        b: dog

  Recursively delete matching keys
Given a sample.yml file of:

        a:
          name: frog
          b:
            name: blog
            age: 12

then

        yq 'del(.. | select(has("name")).name)' sample.yml

will output

        a:
          b:
            age: 12

Divide
Divide behaves differently according to the type of the LHS: * strings: split by the divider * number: arithmetic division

  String split
Given a sample.yml file of:

        a: cat_meow
        b: _

then

        yq '.c = .a / .b' sample.yml

will output

        a: cat_meow
        b: _
        c:
          - cat
          - meow

  Number division
The result during division is calculated as a float

Given a sample.yml file of:

        a: 12
        b: 2.5

then

        yq '.a = .a / .b' sample.yml

will output

        a: 4.8
        b: 2.5

  Number division by zero
Dividing by zero results in +Inf or -Inf

Given a sample.yml file of:

        a: 1
        b: -1

then

        yq '.a = .a / 0 | .b = .b / 0' sample.yml

will output

        a: !!float +Inf
        b: !!float -Inf

Document Index
Use the documentIndex operator (or the di shorthand) to select nodes of a particular document.

  Retrieve a document index
Given a sample.yml file of:

        a: cat
        ---
        a: frog

then

        yq '.a | document_index' sample.yml

will output

        0
        ---
        1

  Retrieve a document index, shorthand
Given a sample.yml file of:

        a: cat
        ---
        a: frog

then

        yq '.a | di' sample.yml

will output

        0
        ---
        1

  Filter by document index
Given a sample.yml file of:

        a: cat
        ---
        a: frog

then

        yq 'select(document_index == 1)' sample.yml

will output

        a: frog

  Filter by document index shorthand
Given a sample.yml file of:

        a: cat
        ---
        a: frog

then

        yq 'select(di == 1)' sample.yml

will output

        a: frog

  Print Document Index with matches
Given a sample.yml file of:

        a: cat
        ---
        a: frog

then

        yq '.a | ({"match": ., "doc": document_index})' sample.yml

will output

        match: cat
        doc: 0
        ---
        match: frog
        doc: 1

Encoder / Decoder
Encode operators will take the piped in object structure and encode it as a string in the desired format. The decode operators do the opposite, they take a formatted string and decode it into the relevant object structure.

Note that you can optionally pass an indent value to the encode functions (see below).

These operators are useful to process yaml documents that have stringified embedded yaml/json/props in them.

Format | Decode (from string) | Encode (to string)
Yaml | from_yaml/@yamld | to_yaml(i)/@yaml
JSON | from_json/@jsond | to_json(i)/@json
Properties | from_props/@propsd | to_props/@props
CSV | from_csv/@csvd | to_csv/@csv
TSV | from_tsv/@tsvd | to_tsv/@tsv
XML | from_xml/@xmld | to_xml(i)/@xml
Base64 | @base64d | @base64
URI | @urid | @uri
Shell |  | @sh

See CSV and TSV documentation (https://mikefarah.gitbook.io/yq/usage/csv-tsv) for accepted formats.

XML uses the --xml-attribute-prefix and xml-content-name flags to identify attributes and content fields.

Base64 assumes rfc4648 (https://rfc-editor.org/rfc/rfc4648.html) encoding. Encoding and decoding both assume that the content is a utf-8 string and not binary content.

  Encode value as json string
Given a sample.yml file of:

        a:
          cool: thing

then

        yq '.b = (.a | to_json)' sample.yml

will output

        a:
          cool: thing
        b: |
          {
            "cool": "thing"
          }

  Encode value as json string, on one line
Pass in a 0 indent to print json on a single line.

Given a sample.yml file of:

        a:
          cool: thing

then

        yq '.b = (.a | to_json(0))' sample.yml

will output

        a:
          cool: thing
        b: '{"cool":"thing"}'

  Encode value as json string, on one line shorthand
Pass in a 0 indent to print json on a single line.

Given a sample.yml file of:

        a:
          cool: thing

then

        yq '.b = (.a | @json)' sample.yml

will output

        a:
          cool: thing
        b: '{"cool":"thing"}'

  Decode a json encoded string
Keep in mind JSON is a subset of YAML. If you want idiomatic yaml, pipe through the style operator to clear out the JSON styling.

Given a sample.yml file of:

        a: '{"cool":"thing"}'

then

        yq '.a | from_json | ... style=""' sample.yml

will output

        cool: thing

  Encode value as props string
Given a sample.yml file of:

        a:
          cool: thing

then

        yq '.b = (.a | @props)' sample.yml

will output

        a:
          cool: thing
        b: |
          cool = thing

  Decode props encoded string
Given a sample.yml file of:

        a: |-
          cats=great
          dogs=cool as well

then

        yq '.a |= @propsd' sample.yml

will output

        a:
          cats: great
          dogs: cool as well

  Decode csv encoded string
Given a sample.yml file of:

        a: |-
          cats,dogs
          great,cool as well

then

        yq '.a |= @csvd' sample.yml

will output

        a:
          - cats: great
            dogs: cool as well

  Decode tsv encoded string
Given a sample.yml file of:

        a: |-
          cats  dogs
          great cool as well

then

        yq '.a |= @tsvd' sample.yml

will output

        a:
          - cats: great
            dogs: cool as well

  Encode value as yaml string
Indent defaults to 2

Given a sample.yml file of:

        a:
          cool:
            bob: dylan

then

        yq '.b = (.a | to_yaml)' sample.yml

will output

        a:
          cool:
            bob: dylan
        b: |
          cool:
            bob: dylan

  Encode value as yaml string, with custom indentation
You can specify the indentation level as the first parameter.

Given a sample.yml file of:

        a:
          cool:
            bob: dylan

then

        yq '.b = (.a | to_yaml(8))' sample.yml

will output

        a:
          cool:
            bob: dylan
        b: |
          cool:
                  bob: dylan

  Decode a yaml encoded string
Given a sample.yml file of:

        a: 'foo: bar'

then

        yq '.b = (.a | from_yaml)' sample.yml

will output

        a: 'foo: bar'
        b:
          foo: bar

  Update a multiline encoded yaml string
Given a sample.yml file of:

        a: |
          foo: bar
          baz: dog

then

        yq '.a |= (from_yaml | .foo = "cat" | to_yaml)' sample.yml

will output

        a: |
          foo: cat
          baz: dog

  Update a single line encoded yaml string
Given a sample.yml file of:

        a: 'foo: bar'

then

        yq '.a |= (from_yaml | .foo = "cat" | to_yaml)' sample.yml

will output

        a: 'foo: cat'

  Encode array of scalars as csv string
Scalars are strings, numbers and booleans.

Given a sample.yml file of:

        - cat
        - thing1,thing2
        - true
        - 3.40

then

        yq '@csv' sample.yml

will output

        cat,"thing1,thing2",true,3.40

  Encode array of arrays as csv string
Given a sample.yml file of:

        - - cat
          - thing1,thing2
          - true
          - 3.40
        - - dog
          - thing3
          - false
          - 12

then

        yq '@csv' sample.yml

will output

        cat,"thing1,thing2",true,3.40
        dog,thing3,false,12

  Encode array of arrays as tsv string
Scalars are strings, numbers and booleans.

Given a sample.yml file of:

        - - cat
          - thing1,thing2
          - true
          - 3.40
        - - dog
          - thing3
          - false
          - 12

then

        yq '@tsv' sample.yml

will output

        cat thing1,thing2   true    3.40
        dog thing3  false   12

  Encode value as xml string
Given a sample.yml file of:

        a:
          cool:
            foo: bar
            +@id: hi

then

        yq '.a | to_xml' sample.yml

will output

        <cool id="hi">
          <foo>bar</foo>
        </cool>

  Encode value as xml string on a single line
Given a sample.yml file of:

        a:
          cool:
            foo: bar
            +@id: hi

then

        yq '.a | @xml' sample.yml

will output

        <cool id="hi"><foo>bar</foo></cool>

  Encode value as xml string with custom indentation
Given a sample.yml file of:

        a:
          cool:
            foo: bar
            +@id: hi

then

        yq '{"cat": .a | to_xml(1)}' sample.yml

will output

        cat: |
          <cool id="hi">
           <foo>bar</foo>
          </cool>

  Decode a xml encoded string
Given a sample.yml file of:

        a: <foo>bar</foo>

then

        yq '.b = (.a | from_xml)' sample.yml

will output

        a: <foo>bar</foo>
        b:
          foo: bar

  Encode a string to base64
Given a sample.yml file of:

        coolData: a special string

then

        yq '.coolData | @base64' sample.yml

will output

        YSBzcGVjaWFsIHN0cmluZw==

  Encode a yaml document to base64
Pipe through @yaml first to convert to a string, then use @base64 to encode it.

Given a sample.yml file of:

        a: apple

then

        yq '@yaml | @base64' sample.yml

will output

        YTogYXBwbGUK

  Encode a string to uri
Given a sample.yml file of:

        coolData: this has & special () characters *

then

        yq '.coolData | @uri' sample.yml

will output

        this+has+%26+special+%28%29+characters+%2A

  Decode a URI to a string
Given a sample.yml file of:

        this+has+%26+special+%28%29+characters+%2A

then

        yq '@urid' sample.yml

will output

        this has & special () characters *

  Encode a string to sh
Sh/Bash friendly string

Given a sample.yml file of:

        coolData: strings with spaces and a 'quote'

then

        yq '.coolData | @sh' sample.yml

will output

        strings' with spaces and a '\'quote\'

  Decode a base64 encoded string
Decoded data is assumed to be a string.

Given a sample.yml file of:

        coolData: V29ya3Mgd2l0aCBVVEYtMTYg8J+Yig==

then

        yq '.coolData | @base64d' sample.yml

will output

        Works with UTF-16 😊

  Decode a base64 encoded yaml document
Pipe through from_yaml to parse the decoded base64 string as a yaml document.

Given a sample.yml file of:

        coolData: YTogYXBwbGUK

then

        yq '.coolData |= (@base64d | from_yaml)' sample.yml

will output

        coolData:
          a: apple

Entries
Similar to the same named functions in jq these functions convert to/from an object and an array of key-value pairs. This is most useful for performing operations on keys of maps.

  to_entries Map
Given a sample.yml file of:

        a: 1
        b: 2

then

        yq 'to_entries' sample.yml

will output

        - key: a
          value: 1
        - key: b
          value: 2

  to_entries Array
Given a sample.yml file of:

        - a
        - b

then

        yq 'to_entries' sample.yml

will output

        - key: 0
          value: a
        - key: 1
          value: b

  to_entries null
Given a sample.yml file of:

        null

then

        yq 'to_entries' sample.yml

will output

  from_entries map
Given a sample.yml file of:

        a: 1
        b: 2

then

        yq 'to_entries | from_entries' sample.yml

will output

        a: 1
        b: 2

  from_entries with numeric key indices
from_entries always creates a map, even for numeric keys

Given a sample.yml file of:

        - a
        - b

then

        yq 'to_entries | from_entries' sample.yml

will output

        0: a
        1: b

  Use with_entries to update keys
Given a sample.yml file of:

        a: 1
        b: 2

then

        yq 'with_entries(.key |= "KEY_" + .)' sample.yml

will output

        KEY_a: 1
        KEY_b: 2

  Custom sort map keys
Use to_entries to convert to an array of key/value pairs, sort the array using sort/sort_by/etc, and convert it back.

Given a sample.yml file of:

        a: 1
        c: 3
        b: 2

then

        yq 'to_entries | sort_by(.key) | reverse | from_entries' sample.yml

will output

        c: 3
        b: 2
        a: 1

  Use with_entries to filter the map
Given a sample.yml file of:

        a:
          b: bird
        c:
          d: dog

then

        yq 'with_entries(select(.value | has("b")))' sample.yml

will output

        a:
          b: bird

Env Variable Operators
These operators are used to handle environment variables usage in expressions and documents. While environment variables can, of course, be passed in via your CLI with string interpolation, this often comes with complex quote escaping and can be tricky to write and read.

There are three operators:

•   env which takes a single environment variable name and parse the variable as a yaml node (be it a map, array, string, number of boolean)

•   strenv which also takes a single environment variable name, and always parses the variable as a string.

•   envsubst which you pipe strings into and it interpolates environment variables in strings using envsubst (https://github.com/a8m/envsubst) .

  EnvSubst Options
You can optionally pass envsubst any of the following options:

•   nu: NoUnset, this will fail if there are any referenced variables that are not set

•   ne: NoEmpty, this will fail if there are any referenced variables that are empty

•   ff: FailFast, this will abort on the first failure (rather than collect all the errors)

E.g: envsubst(ne, ff) will fail on the first empty variable.

See Imposing Restrictions (https://github.com/a8m/envsubst#imposing-restrictions) in the envsubst documentation for more information, and below for examples.

  Tip
To replace environment variables across all values in a document, envsubst can be used with the recursive descent operator as follows:

        yq '(.. | select(tag == "!!str")) |= envsubst' file.yaml

  Read string environment variable
Running

        myenv="cat meow" yq --null-input '.a = env(myenv)'

will output

        a: cat meow

  Read boolean environment variable
Running

        myenv="true" yq --null-input '.a = env(myenv)'

will output

        a: true

  Read numeric environment variable
Running

        myenv="12" yq --null-input '.a = env(myenv)'

will output

        a: 12

  Read yaml environment variable
Running

        myenv="{b: fish}" yq --null-input '.a = env(myenv)'

will output

        a: {b: fish}

  Read boolean environment variable as a string
Running

        myenv="true" yq --null-input '.a = strenv(myenv)'

will output

        a: "true"

  Read numeric environment variable as a string
Running

        myenv="12" yq --null-input '.a = strenv(myenv)'

will output

        a: "12"

  Dynamically update a path from an environment variable
The env variable can be any valid yq expression.

Given a sample.yml file of:

        a:
          b:
            - name: dog
            - name: cat

then

        pathEnv=".a.b[0].name"  valueEnv="moo" yq 'eval(strenv(pathEnv)) = strenv(valueEnv)' sample.yml

will output

        a:
          b:
            - name: moo
            - name: cat

  Dynamic key lookup with environment variable
Given a sample.yml file of:

        cat: meow
        dog: woof

then

        myenv="cat" yq '.[env(myenv)]' sample.yml

will output

        meow

  Replace strings with envsubst
Running

        myenv="cat" yq --null-input '"the ${myenv} meows" | envsubst'

will output

        the cat meows

  Replace strings with envsubst, missing variables
Running

        yq --null-input '"the ${myenvnonexisting} meows" | envsubst'

will output

        the  meows

  Replace strings with envsubst(nu), missing variables
(nu) not unset, will fail if there are unset (missing) variables

Running

        yq --null-input '"the ${myenvnonexisting} meows" | envsubst(nu)'

will output

        Error: variable ${myenvnonexisting} not set

  Replace strings with envsubst(ne), missing variables
(ne) not empty, only validates set variables

Running

        yq --null-input '"the ${myenvnonexisting} meows" | envsubst(ne)'

will output

        the  meows

  Replace strings with envsubst(ne), empty variable
(ne) not empty, will fail if a references variable is empty

Running

        myenv="" yq --null-input '"the ${myenv} meows" | envsubst(ne)'

will output

        Error: variable ${myenv} set but empty

  Replace strings with envsubst, missing variables with defaults
Running

        yq --null-input '"the ${myenvnonexisting-dog} meows" | envsubst'

will output

        the dog meows

  Replace strings with envsubst(nu), missing variables with defaults
Having a default specified skips over the missing variable.

Running

        yq --null-input '"the ${myenvnonexisting-dog} meows" | envsubst(nu)'

will output

        the dog meows

  Replace strings with envsubst(ne), missing variables with defaults
Fails, because the variable is explicitly set to blank.

Running

        myEmptyEnv="" yq --null-input '"the ${myEmptyEnv-dog} meows" | envsubst(ne)'

will output

        Error: variable ${myEmptyEnv} set but empty

  Replace string environment variable in document
Given a sample.yml file of:

        v: ${myenv}

then

        myenv="cat meow" yq '.v |= envsubst' sample.yml

will output

        v: cat meow

  (Default) Return all envsubst errors
By default, all errors are returned at once.

Running

        yq --null-input '"the ${notThere} ${alsoNotThere}" | envsubst(nu)'

will output

        Error: variable ${notThere} not set
        variable ${alsoNotThere} not set

  Fail fast, return the first envsubst error (and abort)
Running

        yq --null-input '"the ${notThere} ${alsoNotThere}" | envsubst(nu,ff)'

will output

        Error: variable ${notThere} not set

Equals / Not Equals
This is a boolean operator that will return true if the LHS is equal to the RHS and false otherwise.

        .a == .b

It is most often used with the select operator to find particular nodes:

        select(.a == .b)

The not equals != operator returns false if the LHS is equal to the RHS.

  Related Operators

•   comparison (>=, < etc) operators here (https://mikefarah.gitbook.io/yq/operators/compare)

•   boolean operators (and, or, any etc) here (https://mikefarah.gitbook.io/yq/operators/boolean-operators)

•   select operator here (https://mikefarah.gitbook.io/yq/operators/select)

  Match string
Given a sample.yml file of:

        - cat
        - goat
        - dog

then

        yq '.[] | (. == "*at")' sample.yml

will output

        true
        true
        false

  Don’t match string
Given a sample.yml file of:

        - cat
        - goat
        - dog

then

        yq '.[] | (. != "*at")' sample.yml

will output

        false
        false
        true

  Match number
Given a sample.yml file of:

        - 3
        - 4
        - 5

then

        yq '.[] | (. == 4)' sample.yml

will output

        false
        true
        false

  Don’t match number
Given a sample.yml file of:

        - 3
        - 4
        - 5

then

        yq '.[] | (. != 4)' sample.yml

will output

        true
        false
        true

  Match nulls
Running

        yq --null-input 'null == ~'

will output

        true

  Non existent key doesn’t equal a value
Given a sample.yml file of:

        a: frog

then

        yq 'select(.b != "thing")' sample.yml

will output

        a: frog

  Two non existent keys are equal
Given a sample.yml file of:

        a: frog

then

        yq 'select(.b == .c)' sample.yml

will output

        a: frog

Error
Use this operation to short-circuit expressions. Useful for validation.

  Validate a particular value
Given a sample.yml file of:

        a: hello

then

        yq 'select(.a == "howdy") or error(".a [" + .a + "] is not howdy!")' sample.yml

will output

        Error: .a [hello] is not howdy!

  Validate the environment variable is a number - invalid
Running

        numberOfCats="please" yq --null-input 'env(numberOfCats) | select(tag == "!!int") or error("numberOfCats is not a number :(")'

will output

        Error: numberOfCats is not a number :(

  Validate the environment variable is a number - valid
with can be a convenient way of encapsulating validation.

Given a sample.yml file of:

        name: Bob
        favouriteAnimal: cat

then

        numberOfCats="3" yq '
            with(env(numberOfCats); select(tag == "!!int") or error("numberOfCats is not a number :(")) |
            .numPets = env(numberOfCats)
        ' sample.yml

will output

        name: Bob
        favouriteAnimal: cat
        numPets: 3

Eval
Use eval to dynamically process an expression - for instance from an environment variable.

eval takes a single argument, and evaluates that as a yq expression. Any valid expression can be used, be it a path .a.b.c | select(. == "cat"), or an update .a.b.c = "gogo".

Tip: This can be a useful way to parameterise complex scripts.

  Dynamically evaluate a path
Given a sample.yml file of:

        pathExp: .a.b[] | select(.name == "cat")
        a:
          b:
            - name: dog
            - name: cat

then

        yq 'eval(.pathExp)' sample.yml

will output

        name: cat

  Dynamically update a path from an environment variable
The env variable can be any valid yq expression.

Given a sample.yml file of:

        a:
          b:
            - name: dog
            - name: cat

then

        pathEnv=".a.b[0].name"  valueEnv="moo" yq 'eval(strenv(pathEnv)) = strenv(valueEnv)' sample.yml

will output

        a:
          b:
            - name: moo
            - name: cat

File Operators
File operators are most often used with merge when needing to merge specific files together. Note that when doing this, you will need to use eval-all to ensure all yaml documents are loaded into memory before performing the merge (as opposed to eval which runs the expression once per document).

Note that the fileIndex operator has a short alias of fi.

  Merging files
Note the use of eval-all to ensure all documents are loaded into memory.

        yq eval-all 'select(fi == 0) * select(filename == "file2.yaml")' file1.yaml file2.yaml

  Get filename
Given a sample.yml file of:

        a: cat

then

        yq 'filename' sample.yml

will output

        sample.yml

  Get file index
Given a sample.yml file of:

        a: cat

then

        yq 'file_index' sample.yml

will output

        0

  Get file indices of multiple documents
Given a sample.yml file of:

        a: cat

And another sample another.yml file of:

        a: cat

then

        yq eval-all 'file_index' sample.yml another.yml

will output

        0
        1

  Get file index alias
Given a sample.yml file of:

        a: cat

then

        yq 'fi' sample.yml

will output

        0

Filter
Filters an array (or map values) by the expression given. Equivalent to doing map(select(exp)).

  Filter array
Given a sample.yml file of:

        - 1
        - 2
        - 3

then

        yq 'filter(. < 3)' sample.yml

will output

        - 1
        - 2

  Filter map values
Given a sample.yml file of:

        c:
          things: cool
          frog: yes
        d:
          things: hot
          frog: false

then

        yq 'filter(.things == "cool")' sample.yml

will output

        - things: cool
          frog: yes

Flatten
This recursively flattens arrays.

  Flatten
Recursively flattens all arrays

Given a sample.yml file of:

        - 1
        - - 2
        - - - 3

then

        yq 'flatten' sample.yml

will output

        - 1
        - 2
        - 3

  Flatten with depth of one
Given a sample.yml file of:

        - 1
        - - 2
        - - - 3

then

        yq 'flatten(1)' sample.yml

will output

        - 1
        - 2
        - - 3

  Flatten empty array
Given a sample.yml file of:

        - []

then

        yq 'flatten' sample.yml

will output

        []

  Flatten array of objects
Given a sample.yml file of:

        - foo: bar
        - - foo: baz

then

        yq 'flatten' sample.yml

will output

        - foo: bar
        - foo: baz

Group By
This is used to group items in an array by an expression.

  Group by field
Given a sample.yml file of:

        - foo: 1
          bar: 10
        - foo: 3
          bar: 100
        - foo: 1
          bar: 1

then

        yq 'group_by(.foo)' sample.yml

will output

        - - foo: 1
            bar: 10
          - foo: 1
            bar: 1
        - - foo: 3
            bar: 100

  Group by field, with nulls
Given a sample.yml file of:

        - cat: dog
        - foo: 1
          bar: 10
        - foo: 3
          bar: 100
        - no: foo for you
        - foo: 1
          bar: 1

then

        yq 'group_by(.foo)' sample.yml

will output

        - - cat: dog
          - no: foo for you
        - - foo: 1
            bar: 10
          - foo: 1
            bar: 1
        - - foo: 3
            bar: 100

Has
This operation returns true if the key exists in a map (or index in an array), false otherwise.

  Has map key
Given a sample.yml file of:

        - a: yes
        - a: ~
        - a:
        - b: nope

then

        yq '.[] | has("a")' sample.yml

will output

        true
        true
        true
        false

  Select, checking for existence of deep paths
Simply pipe in parent expressions into has

Given a sample.yml file of:

        - a:
            b:
              c: cat
        - a:
            b:
              d: dog

then

        yq '.[] | select(.a.b | has("c"))' sample.yml

will output

        a:
          b:
            c: cat

  Has array index
Given a sample.yml file of:

        - []
        - [1]
        - [1, 2]
        - [1, null]
        - [1, 2, 3]

then

        yq '.[] | has(1)' sample.yml

will output

        false
        false
        true
        true
        true

Keys
Use the keys operator to return map keys or array indices.

  Map keys
Given a sample.yml file of:

        dog: woof
        cat: meow

then

        yq 'keys' sample.yml

will output

        - dog
        - cat

  Array keys
Given a sample.yml file of:

        - apple
        - banana

then

        yq 'keys' sample.yml

will output

        - 0
        - 1

  Retrieve array key
Given a sample.yml file of:

        - 1
        - 2
        - 3

then

        yq '.[1] | key' sample.yml

will output

        1

  Retrieve map key
Given a sample.yml file of:

        a: thing

then

        yq '.a | key' sample.yml

will output

        a

  No key
Given a sample.yml file of:

        {}

then

        yq 'key' sample.yml

will output

  Update map key
Given a sample.yml file of:

        a:
          x: 3
          y: 4

then

        yq '(.a.x | key) = "meow"' sample.yml

will output

        a:
          meow: 3
          y: 4

  Get comment from map key
Given a sample.yml file of:

        a:
          # comment on key
          x: 3
          y: 4

then

        yq '.a.x | key | headComment' sample.yml

will output

        comment on key

  Check node is a key
Given a sample.yml file of:

        a:
          b:
            - cat
          c: frog

then

        yq '[... | { "p": path | join("."), "isKey": is_key, "tag": tag }]' sample.yml

will output

        - p: ""
          isKey: false
          tag: '!!map'
        - p: a
          isKey: true
          tag: '!!str'
        - p: a
          isKey: false
          tag: '!!map'
        - p: a.b
          isKey: true
          tag: '!!str'
        - p: a.b
          isKey: false
          tag: '!!seq'
        - p: a.b.0
          isKey: false
          tag: '!!str'
        - p: a.c
          isKey: true
          tag: '!!str'
        - p: a.c
          isKey: false
          tag: '!!str'

  Get kind
Given a sample.yml file of:

        a: cat
        b: 5
        c: 3.2
        e: true
        f: []
        g: {}
        h: null

then

        yq '.. | kind' sample.yml

will output

        map
        scalar
        scalar
        scalar
        scalar
        seq
        map
        scalar

  Get kind, ignores custom tags
Unlike tag, kind is not affected by custom tags.

Given a sample.yml file of:

        a: !!thing cat
        b: !!foo {}
        c: !!bar []

then

        yq '.. | kind' sample.yml

will output

        map
        scalar
        map
        seq

  Add comments only to scalars
An example of how you can use kind

Given a sample.yml file of:

        a:
          b: 5
          c: 3.2
        e: true
        f: []
        g: {}
        h: null

then

        yq '(.. | select(kind == "scalar")) line_comment = "this is a scalar"' sample.yml

will output

        a:
          b: 5 # this is a scalar
          c: 3.2 # this is a scalar
        e: true # this is a scalar
        f: []
        g: {}
        h: null # this is a scalar

Length
Returns the lengths of the nodes. Length is defined according to the type of the node.

  String length
returns length of string

Given a sample.yml file of:

        a: cat

then

        yq '.a | length' sample.yml

will output

        3

  null length
Given a sample.yml file of:

        a: null

then

        yq '.a | length' sample.yml

will output

        0

  Map length
returns number of entries

Given a sample.yml file of:

        a: cat
        c: dog

then

        yq 'length' sample.yml

will output

        2

  Array length
returns number of elements

Given a sample.yml file of:

        - 2
        - 4
        - 6
        - 8

then

        yq 'length' sample.yml

will output

        4

Line
Returns the line of the matching node. Starts from 1, 0 indicates there was no line data.

  Returns line of value node
Given a sample.yml file of:

        a: cat
        b:
          c: cat

then

        yq '.b | line' sample.yml

will output

        3

  Returns line of key node
Pipe through the key operator to get the line of the key

Given a sample.yml file of:

        a: cat
        b:
          c: cat

then

        yq '.b | key | line' sample.yml

will output

        2

  First line is 1
Given a sample.yml file of:

        a: cat

then

        yq '.a | line' sample.yml

will output

        1

  No line data is 0
Running

        yq --null-input '{"a": "new entry"} | line'

will output

        0

Load
The load operators allows you to load in content from another file.

Note that you can use string operators like + and sub to modify the value in the yaml file to a path that exists in your system.

You can load files of the following supported types:

Format | Load Operator
Yaml | load
XML | load_xml
Properties | load_props
Plain String | load_str
Base64 | load_base64

Note that load_base64 only works for base64 encoded utf-8 strings.

  Samples files for tests:

  yaml
../../examples/thing.yml:

        a: apple is included
        b: cool

  xml
small.xml:

        <this>is some xml</this>

  properties
small.properties:

        this.is = a properties file

  base64
base64.txt:

        bXkgc2VjcmV0IGNoaWxsaSByZWNpcGUgaXMuLi4u

  Simple example
Given a sample.yml file of:

        myFile: ../../examples/thing.yml

then

        yq 'load(.myFile)' sample.yml

will output

        a: apple is included
        b: cool.

  Replace node with referenced file
Note that you can modify the filename in the load operator if needed.

Given a sample.yml file of:

        something:
          file: thing.yml

then

        yq '.something |= load("../../examples/" + .file)' sample.yml

will output

        something:
          a: apple is included
          b: cool.

  Replace all nodes with referenced file
Recursively match all the nodes (..) and then filter the ones that have a `file' attribute.

Given a sample.yml file of:

        something:
          file: thing.yml
        over:
          here:
            - file: thing.yml

then

        yq '(.. | select(has("file"))) |= load("../../examples/" + .file)' sample.yml

will output

        something:
          a: apple is included
          b: cool.
        over:
          here:
            - a: apple is included
              b: cool.

  Replace node with referenced file as string
This will work for any text based file

Given a sample.yml file of:

        something:
          file: thing.yml

then

        yq '.something |= load_str("../../examples/" + .file)' sample.yml

will output

        something: |-
          a: apple is included
          b: cool.

  Load from XML
Given a sample.yml file of:

        cool: things

then

        yq '.more_stuff = load_xml("../../examples/small.xml")' sample.yml

will output

        cool: things
        more_stuff:
          this: is some xml

  Load from Properties
Given a sample.yml file of:

        cool: things

then

        yq '.more_stuff = load_props("../../examples/small.properties")' sample.yml

will output

        cool: things
        more_stuff:
          this:
            is: a properties file

  Merge from properties
This can be used as a convenient way to update a yaml document

Given a sample.yml file of:

        this:
          is: from yaml
          cool: ay

then

        yq '. *= load_props("../../examples/small.properties")' sample.yml

will output

        this:
          is: a properties file
          cool: ay

  Load from base64 encoded file
Given a sample.yml file of:

        cool: things

then

        yq '.more_stuff = load_base64("../../examples/base64.txt")' sample.yml

will output

        cool: things
        more_stuff: my secret chilli recipe is....

Map
Maps values of an array. Use map_values to map values of an object.

  Map array
Given a sample.yml file of:

        - 1
        - 2
        - 3

then

        yq 'map(. + 1)' sample.yml

will output

        - 2
        - 3
        - 4

  Map object values
Given a sample.yml file of:

        a: 1
        b: 2
        c: 3

then

        yq 'map_values(. + 1)' sample.yml

will output

        a: 2
        b: 3
        c: 4

Max
Computes the maximum among an incoming sequence of scalar values.

  Maximum int
Given a sample.yml file of:

        - 99
        - 16
        - 12
        - 6
        - 66

then

        yq 'max' sample.yml

will output

        99

  Maximum string
Given a sample.yml file of:

        - foo
        - bar
        - baz

then

        yq 'max' sample.yml

will output

        foo

  Maximum of empty
Given a sample.yml file of:

        []

then

        yq 'max' sample.yml

will output

Min
Computes the minimum among an incoming sequence of scalar values.

  Minimum int
Given a sample.yml file of:

        - 99
        - 16
        - 12
        - 6
        - 66

then

        yq 'min' sample.yml

will output

        6

  Minimum string
Given a sample.yml file of:

        - foo
        - bar
        - baz

then

        yq 'min' sample.yml

will output

        bar

  Minimum of empty
Given a sample.yml file of:

        []

then

        yq 'min' sample.yml

will output

Modulo
Arithmetic modulo operator, returns the remainder from dividing two numbers.

  Number modulo - int
If the lhs and rhs are ints then the expression will be calculated with ints.

Given a sample.yml file of:

        a: 13
        b: 2

then

        yq '.a = .a % .b' sample.yml

will output

        a: 1
        b: 2

  Number modulo - float
If the lhs or rhs are floats then the expression will be calculated with floats.

Given a sample.yml file of:

        a: 12
        b: 2.5

then

        yq '.a = .a % .b' sample.yml

will output

        a: !!float 2
        b: 2.5

  Number modulo - int by zero
If the lhs is an int and rhs is a 0 the result is an error.

Given a sample.yml file of:

        a: 1
        b: 0

then

        yq '.a = .a % .b' sample.yml

will output

        Error: cannot modulo by 0

  Number modulo - float by zero
If the lhs is a float and rhs is a 0 the result is NaN.

Given a sample.yml file of:

        a: 1.1
        b: 0

then

        yq '.a = .a % .b' sample.yml

will output

        a: !!float NaN
        b: 0

Multiply (Merge)
Like the multiple operator in jq, depending on the operands, this multiply operator will do different things. Currently numbers, arrays and objects are supported.

  Objects and arrays - merging
Objects are merged deeply matching on matching keys. By default, array values override and are not deeply merged.

You can use the add operator +, to shallow merge objects, see more info here (https://mikefarah.gitbook.io/yq/operators/add) .

Note that when merging objects, this operator returns the merged object (not the parent). This will be clearer in the examples below.

  Merge Flags
You can control how objects are merged by using one or more of the following flags. Multiple flags can be used together, e.g. .a *+? .b. See examples below

•   + append arrays

•   d deeply merge arrays

•   ? only merge existing fields

•   n only merge new fields

•   c clobber custom tags

To perform a shallow merge only, use the add operator +, see more info here (https://mikefarah.gitbook.io/yq/operators/add) .

  Merge two files together
This uses the load operator to merge file2 into file1.

        yq '. *= load("file2.yml")' file1.yml

  Merging all files
Note the use of eval-all to ensure all documents are loaded into memory.

        yq eval-all '. as $item ireduce ({}; . * $item )' *.yml

Merging complex arrays together by a key field
By default - yq merge is naive. It merges maps when they match the key name, and arrays are merged either by appending them together, or merging the entries by their position in the array.

For more complex array merging (e.g. merging items that match on a certain key) please see the example here (https://mikefarah.gitbook.io/yq/operators/multiply-merge#merge-arrays-of-objects-together-matching-on-a-key)

  Multiply integers
Given a sample.yml file of:

        a: 3
        b: 4

then

        yq '.a *= .b' sample.yml

will output

        a: 12
        b: 4

  Multiply string node X int
Given a sample.yml file of:

        b: banana

then

        yq '.b * 4' sample.yml

will output

        bananabananabananabanana

  Multiply int X string node
Given a sample.yml file of:

        b: banana

then

        yq '4 * .b' sample.yml

will output

        bananabananabananabanana

  Multiply string X int node
Given a sample.yml file of:

        n: 4

then

        yq '"banana" * .n' sample.yml

will output

        bananabananabananabanana

  Multiply int node X string
Given a sample.yml file of:

        n: 4

then

        yq '.n * "banana"' sample.yml

will output

        bananabananabananabanana

  Merge objects together, returning merged result only
Given a sample.yml file of:

        a:
          field: me
          fieldA: cat
        b:
          field:
            g: wizz
          fieldB: dog

then

        yq '.a * .b' sample.yml

will output

        field:
          g: wizz
        fieldA: cat
        fieldB: dog

  Merge objects together, returning parent object
Given a sample.yml file of:

        a:
          field: me
          fieldA: cat
        b:
          field:
            g: wizz
          fieldB: dog

then

        yq '. * {"a":.b}' sample.yml

will output

        a:
          field:
            g: wizz
          fieldA: cat
          fieldB: dog
        b:
          field:
            g: wizz
          fieldB: dog

  Merge keeps style of LHS
Given a sample.yml file of:

        a: {things: great}
        b:
          also: "me"

then

        yq '. * {"a":.b}' sample.yml

will output

        a: {things: great, also: "me"}
        b:
          also: "me"

  Merge arrays
Given a sample.yml file of:

        a:
          - 1
          - 2
          - 3
        b:
          - 3
          - 4
          - 5

then

        yq '. * {"a":.b}' sample.yml

will output

        a:
          - 3
          - 4
          - 5
        b:
          - 3
          - 4
          - 5

  Merge, only existing fields
Given a sample.yml file of:

        a:
          thing: one
          cat: frog
        b:
          missing: two
          thing: two

then

        yq '.a *? .b' sample.yml

will output

        thing: two
        cat: frog

  Merge, only new fields
Given a sample.yml file of:

        a:
          thing: one
          cat: frog
        b:
          missing: two
          thing: two

then

        yq '.a *n .b' sample.yml

will output

        thing: one
        cat: frog
        missing: two

  Merge, appending arrays
Given a sample.yml file of:

        a:
          array:
            - 1
            - 2
            - animal: dog
          value: coconut
        b:
          array:
            - 3
            - 4
            - animal: cat
          value: banana

then

        yq '.a *+ .b' sample.yml

will output

        array:
          - 1
          - 2
          - animal: dog
          - 3
          - 4
          - animal: cat
        value: banana

  Merge, only existing fields, appending arrays
Given a sample.yml file of:

        a:
          thing:
            - 1
            - 2
        b:
          thing:
            - 3
            - 4
          another:
            - 1

then

        yq '.a *?+ .b' sample.yml

will output

        thing:
          - 1
          - 2
          - 3
          - 4

  Merge, deeply merging arrays
Merging arrays deeply means arrays are merged like objects, with indices as their key. In this case, we merge the first item in the array and do nothing with the second.

Given a sample.yml file of:

        a:
          - name: fred
            age: 12
          - name: bob
            age: 32
        b:
          - name: fred
            age: 34

then

        yq '.a *d .b' sample.yml

will output

        - name: fred
          age: 34
        - name: bob
          age: 32

  Merge arrays of objects together, matching on a key
This is a fairly complex expression - you can use it as is by providing the environment variables as seen in the example below.

It merges in the array provided in the second file into the first - matching on equal keys.

Explanation:

The approach, at a high level, is to reduce into a merged map (keyed by the unique key) and then convert that back into an array.

First the expression will create a map from the arrays keyed by the idPath, the unique field we want to merge by. The reduce operator is merging `({}; . * $item )', so array elements with the matching key will be merged together.

Next, we convert the map back to an array, using reduce again, concatenating all the map values together.

Finally, we set the result of the merged array back into the first doc.

Thanks Kev from stackoverflow (https://stackoverflow.com/a/70109529/1168223)

Given a sample.yml file of:

        myArray:
          - a: apple
            b: appleB
          - a: kiwi
            b: kiwiB
          - a: banana
            b: bananaB
        something: else

And another sample another.yml file of:

        newArray:
          - a: banana
            c: bananaC
          - a: apple
            b: appleB2
          - a: dingo
            c: dingoC

then

        idPath=".a"  originalPath=".myArray"  otherPath=".newArray" yq eval-all '
        (
          (( (eval(strenv(originalPath)) + eval(strenv(otherPath)))  | .[] | {(eval(strenv(idPath))):  .}) as $item ireduce ({}; . * $item )) as $uniqueMap
          | ( $uniqueMap  | to_entries | .[]) as $item ireduce([]; . + $item.value)
        ) as $mergedArray
        | select(fi == 0) | (eval(strenv(originalPath))) = $mergedArray
        ' sample.yml another.yml

will output

        myArray:
          - a: apple
            b: appleB2
          - a: kiwi
            b: kiwiB
          - a: banana
            b: bananaB
            c: bananaC
          - a: dingo
            c: dingoC
        something: else

  Merge to prefix an element
Given a sample.yml file of:

        a: cat
        b: dog

then

        yq '. * {"a": {"c": .a}}' sample.yml

will output

        a:
          c: cat
        b: dog

  Merge with simple aliases
Given a sample.yml file of:

        a: &cat
          c: frog
        b:
          f: *cat
        c:
          g: thongs

then

        yq '.c * .b' sample.yml

will output

        g: thongs
        f: *cat

  Merge copies anchor names
Given a sample.yml file of:

        a:
          c: &cat frog
        b:
          f: *cat
        c:
          g: thongs

then

        yq '.c * .a' sample.yml

will output

        g: thongs
        c: &cat frog

  Merge with merge anchors
Given a sample.yml file of:

        foo: &foo
          a: foo_a
          thing: foo_thing
          c: foo_c
        bar: &bar
          b: bar_b
          thing: bar_thing
          c: bar_c
        foobarList:
          b: foobarList_b
          !!merge <<:
            - *foo
            - *bar
          c: foobarList_c
        foobar:
          c: foobar_c
          !!merge <<: *foo
          thing: foobar_thing

then

        yq '.foobar * .foobarList' sample.yml

will output

        c: foobarList_c
        !!merge <<:
          - *foo
          - *bar
        thing: foobar_thing
        b: foobarList_b

  Custom types: that are really numbers
When custom tags are encountered, yq will try to decode the underlying type.

Given a sample.yml file of:

        a: !horse 2
        b: !goat 3

then

        yq '.a = .a * .b' sample.yml

will output

        a: !horse 6
        b: !goat 3

  Custom types: that are really maps
Custom tags will be maintained.

Given a sample.yml file of:

        a: !horse
          cat: meow
        b: !goat
          dog: woof

then

        yq '.a = .a * .b' sample.yml

will output

        a: !horse
          cat: meow
          dog: woof
        b: !goat
          dog: woof

  Custom types: clobber tags
Use the c option to clobber custom tags. Note that the second tag is now used.

Given a sample.yml file of:

        a: !horse
          cat: meow
        b: !goat
          dog: woof

then

        yq '.a *=c .b' sample.yml

will output

        a: !goat
          cat: meow
          dog: woof
        b: !goat
          dog: woof

  Merging a null with a map
Running

        yq --null-input 'null * {"some": "thing"}'

will output

        some: thing

  Merging a map with null
Running

        yq --null-input '{"some": "thing"} * null'

will output

        some: thing

  Merging a null with an array
Running

        yq --null-input 'null * ["some"]'

will output

        - some

  Merging an array with null
Running

        yq --null-input '["some"] * null'

will output

        - some

Omit
Works like pick, but instead you specify the keys/indices that you don’t want included.

  Omit keys from map
Note that non existent keys are skipped.

Given a sample.yml file of:

        myMap:
          cat: meow
          dog: bark
          thing: hamster
          hamster: squeak

then

        yq '.myMap |= omit(["hamster", "cat", "goat"])' sample.yml

will output

        myMap:
          dog: bark
          thing: hamster

  Omit indices from array
Note that non existent indices are skipped.

Given a sample.yml file of:

        - cat
        - leopard
        - lion

then

        yq 'omit([2, 0, 734, -5])' sample.yml

will output

        - leopard

Parent
Parent simply returns the parent nodes of the matching nodes.

  Simple example
Given a sample.yml file of:

        a:
          nested: cat

then

        yq '.a.nested | parent' sample.yml

will output

        nested: cat

  Parent of nested matches
Given a sample.yml file of:

        a:
          fruit: apple
          name: bob
        b:
          fruit: banana
          name: sam

then

        yq '.. | select(. == "banana") | parent' sample.yml

will output

        fruit: banana
        name: sam

  N-th parent
You can optionally supply the number of levels to go up for the parent, the default being 1.

Given a sample.yml file of:

        a:
          b:
            c: cat

then

        yq '.a.b.c | parent(2)' sample.yml

will output

        b:
          c: cat

  N-th parent - another level
Given a sample.yml file of:

        a:
          b:
            c: cat

then

        yq '.a.b.c | parent(3)' sample.yml

will output

        a:
          b:
            c: cat

  No parent
Given a sample.yml file of:

        {}

then

        yq 'parent' sample.yml

will output

Path
The path operator can be used to get the traversal paths of matching nodes in an expression. The path is returned as an array, which if traversed in order will lead to the matching node.

You can get the key/index of matching nodes by using the path operator to return the path array then piping that through .[-1] to get the last element of that array, the key.

Use setpath to set a value to the path array returned by path, and similarly delpaths for an array of path arrays.

  Map path
Given a sample.yml file of:

        a:
          b: cat

then

        yq '.a.b | path' sample.yml

will output

        - a
        - b

  Get map key
Given a sample.yml file of:

        a:
          b: cat

then

        yq '.a.b | path | .[-1]' sample.yml

will output

        b

  Array path
Given a sample.yml file of:

        a:
          - cat
          - dog

then

        yq '.a.[] | select(. == "dog") | path' sample.yml

will output

        - a
        - 1

  Get array index
Given a sample.yml file of:

        a:
          - cat
          - dog

then

        yq '.a.[] | select(. == "dog") | path | .[-1]' sample.yml

will output

        1

  Print path and value
Given a sample.yml file of:

        a:
          - cat
          - dog
          - frog

then

        yq '.a[] | select(. == "*og") | [{"path":path, "value":.}]' sample.yml

will output

        - path:
            - a
            - 1
          value: dog
        - path:
            - a
            - 2
          value: frog

  Set path
Given a sample.yml file of:

        a:
          b: cat

then

        yq 'setpath(["a", "b"]; "things")' sample.yml

will output

        a:
          b: things

  Set on empty document
Running

        yq --null-input 'setpath(["a", "b"]; "things")'

will output

        a:
          b: things

  Set path to prune deep paths
Like pick but recursive. This uses ireduce to deeply set the selected paths into an empty object.

Given a sample.yml file of:

        parentA: bob
        parentB:
          child1: i am child1
          child2: i am child2
        parentC:
          child1: me child1
          child2: me child2

then

        yq '(.parentB.child2, .parentC.child1) as $i
          ireduce({}; setpath($i | path; $i))' sample.yml

will output

        parentB:
          child2: i am child2
        parentC:
          child1: me child1

  Set array path
Given a sample.yml file of:

        a:
          - cat
          - frog

then

        yq 'setpath(["a", 0]; "things")' sample.yml

will output

        a:
          - things
          - frog

  Set array path empty
Running

        yq --null-input 'setpath(["a", 0]; "things")'

will output

        a:
          - things

  Delete path
Notice delpaths takes an array of paths.

Given a sample.yml file of:

        a:
          b: cat
          c: dog
          d: frog

then

        yq 'delpaths([["a", "c"], ["a", "d"]])' sample.yml

will output

        a:
          b: cat

  Delete array path
Given a sample.yml file of:

        a:
          - cat
          - frog

then

        yq 'delpaths([["a", 0]])' sample.yml

will output

        a:
          - frog

  Delete - wrong parameter
delpaths does not work with a single path array

Given a sample.yml file of:

        a:
          - cat
          - frog

then

        yq 'delpaths(["a", 0])' sample.yml

will output

        Error: DELPATHS: expected entry [0] to be a sequence, but its a !!str. Note that delpaths takes an array of path arrays, e.g. [["a", "b"]]

Pick
Filter a map by the specified list of keys. Map is returned with the key in the order of the pick list.

Similarly, filter an array by the specified list of indices.

  Pick keys from map
Note that the order of the keys matches the pick order and non existent keys are skipped.

Given a sample.yml file of:

        myMap:
          cat: meow
          dog: bark
          thing: hamster
          hamster: squeak

then

        yq '.myMap |= pick(["hamster", "cat", "goat"])' sample.yml

will output

        myMap:
          hamster: squeak
          cat: meow

  Pick indices from array
Note that the order of the indices matches the pick order and non existent indices are skipped.

Given a sample.yml file of:

        - cat
        - leopard
        - lion

then

        yq 'pick([2, 0, 734, -5])' sample.yml

will output

        - lion
        - cat

Pipe
Pipe the results of an expression into another. Like the bash operator.

  Simple Pipe
Given a sample.yml file of:

        a:
          b: cat

then

        yq '.a | .b' sample.yml

will output

        cat

  Multiple updates
Given a sample.yml file of:

        a: cow
        b: sheep
        c: same

then

        yq '.a = "cat" | .b = "dog"' sample.yml

will output

        a: cat
        b: dog
        c: same

Pivot
Emulates the PIVOT function supported by several popular RDBMS systems.

  Pivot a sequence of sequences
Given a sample.yml file of:

        - - foo
          - bar
          - baz
        - - sis
          - boom
          - bah

then

        yq 'pivot' sample.yml

will output

        - - foo
          - sis
        - - bar
          - boom
        - - baz
          - bah

  Pivot sequence of heterogeneous sequences
Missing values are “padded” to null.

Given a sample.yml file of:

        - - foo
          - bar
          - baz
        - - sis
          - boom
          - bah
          - blah

then

        yq 'pivot' sample.yml

will output

        - - foo
          - sis
        - - bar
          - boom
        - - baz
          - bah
        - -
          - blah

  Pivot sequence of maps
Given a sample.yml file of:

        - foo: a
          bar: b
          baz: c
        - foo: x
          bar: y
          baz: z

then

        yq 'pivot' sample.yml

will output

        foo:
          - a
          - x
        bar:
          - b
          - y
        baz:
          - c
          - z

  Pivot sequence of heterogeneous maps
Missing values are “padded” to null.

Given a sample.yml file of:

        - foo: a
          bar: b
          baz: c
        - foo: x
          bar: y
          baz: z
          what: ever

then

        yq 'pivot' sample.yml

will output

        foo:
          - a
          - x
        bar:
          - b
          - y
        baz:
          - c
          - z
        what:
          -
          - ever

Recursive Descent (Glob)
This operator recursively matches (or globs) all children nodes given of a particular element, including that node itself. This is most often used to apply a filter recursively against all matches.

  match values form ..
This will, like the jq equivalent, recursively match all value nodes. Use it to find/manipulate particular values.

For instance to set the style of all value nodes in a yaml doc, excluding map keys:

        yq '.. style= "flow"' file.yaml

  match values and map keys form ...
The also includes map keys in the results set. This is particularly useful in YAML as unlike JSON, map keys can have their own styling and tags and also use anchors and aliases.

For instance to set the style of all nodes in a yaml doc, including the map keys:

        yq '... style= "flow"' file.yaml

  Recurse map (values only)
Given a sample.yml file of:

        a: frog

then

        yq '..' sample.yml

will output

        a: frog
        frog

  Recursively find nodes with keys
Note that this example has wrapped the expression in [] to show that there are two matches returned. You do not have to wrap in [] in your path expression.

Given a sample.yml file of:

        a:
          name: frog
          b:
            name: blog
            age: 12

then

        yq '[.. | select(has("name"))]' sample.yml

will output

        - name: frog
          b:
            name: blog
            age: 12
        - name: blog
          age: 12

  Recursively find nodes with values
Given a sample.yml file of:

        a:
          nameA: frog
          b:
            nameB: frog
            age: 12

then

        yq '.. | select(. == "frog")' sample.yml

will output

        frog
        frog

  Recurse map (values and keys)
Note that the map key appears in the results

Given a sample.yml file of:

        a: frog

then

        yq '...' sample.yml

will output

        a: frog
        a
        frog

  Aliases are not traversed
Given a sample.yml file of:

        a: &cat
          c: frog
        b: *cat

then

        yq '[..]' sample.yml

will output

        - a: &cat
            c: frog
          b: *cat
        - &cat
          c: frog
        - frog
        - *cat

  Merge docs are not traversed
Given a sample.yml file of:

        foo: &foo
          a: foo_a
          thing: foo_thing
          c: foo_c
        bar: &bar
          b: bar_b
          thing: bar_thing
          c: bar_c
        foobarList:
          b: foobarList_b
          !!merge <<:
            - *foo
            - *bar
          c: foobarList_c
        foobar:
          c: foobar_c
          !!merge <<: *foo
          thing: foobar_thing

then

        yq '.foobar | [..]' sample.yml

will output

        - c: foobar_c
          !!merge <<: *foo
          thing: foobar_thing
        - foobar_c
        - *foo
        - foobar_thing

Reduce
Reduce is a powerful way to process a collection of data into a new form.

        <exp> as $<name> ireduce (<init>; <block>)

e.g.

        .[] as $item ireduce (0; . + $item)

On the LHS we are configuring the collection of items that will be reduced <exp> as well as what each element will be called $<name>. Note that the array has been splatted into its individual elements.

On the RHS there is <init>, the starting value of the accumulator and <block>, the expression that will update the accumulator for each element in the collection. Note that within the block expression, . will evaluate to the current value of the accumulator.

  yq vs jq syntax
Reduce syntax in yq is a little different from jq - as yq (currently) isn’t as sophisticated as jq and its only supports infix notation (e.g. a + b, where the operator is in the middle of the two parameters) - where as jq uses a mix of infix notation with prefix notation (e.g. reduce a b is like writing + a b).

To that end, the reduce operator is called ireduce for backwards compatibility if a jq like prefix version of reduce is ever added.

  Sum numbers
Given a sample.yml file of:

        - 10
        - 2
        - 5
        - 3

then

        yq '.[] as $item ireduce (0; . + $item)' sample.yml

will output

        20

  Merge all yaml files together
Given a sample.yml file of:

        a: cat

And another sample another.yml file of:

        b: dog

then

        yq eval-all '. as $item ireduce ({}; . * $item )' sample.yml another.yml

will output

        a: cat
        b: dog

  Convert an array to an object
Given a sample.yml file of:

        - name: Cathy
          has: apples
        - name: Bob
          has: bananas

then

        yq '.[] as $item ireduce ({}; .[$item | .name] = ($item | .has) )' sample.yml

will output

        Cathy: apples
        Bob: bananas

Reverse
Reverses the order of the items in an array

  Reverse
Given a sample.yml file of:

        - 1
        - 2
        - 3

then

        yq 'reverse' sample.yml

will output

        - 3
        - 2
        - 1

  Sort descending by string field
Use sort with reverse to sort in descending order.

Given a sample.yml file of:

        - a: banana
        - a: cat
        - a: apple

then

        yq 'sort_by(.a) | reverse' sample.yml

will output

        - a: cat
        - a: banana
        - a: apple

Select
Select is used to filter arrays and maps by a boolean expression.

  Related Operators

•   equals / not equals (==, !=) operators here (https://mikefarah.gitbook.io/yq/operators/equals)

•   comparison (>=, < etc) operators here (https://mikefarah.gitbook.io/yq/operators/compare)

•   boolean operators (and, or, any etc) here (https://mikefarah.gitbook.io/yq/operators/boolean-operators)

  Select elements from array using wildcard prefix
Given a sample.yml file of:

        - cat
        - goat
        - dog

then

        yq '.[] | select(. == "*at")' sample.yml

will output

        cat
        goat

  Select elements from array using wildcard suffix
Given a sample.yml file of:

        - go-kart
        - goat
        - dog

then

        yq '.[] | select(. == "go*")' sample.yml

will output

        go-kart
        goat

  Select elements from array using wildcard prefix and suffix
Given a sample.yml file of:

        - ago
        - go
        - meow
        - going

then

        yq '.[] | select(. == "*go*")' sample.yml

will output

        ago
        go
        going

  Select elements from array with regular expression
See more regular expression examples under the string operator docs (https://mikefarah.gitbook.io/yq/operators/string-operators) .

Given a sample.yml file of:

        - this_0
        - not_this
        - nor_0_this
        - thisTo_4

then

        yq '.[] | select(test("[a-zA-Z]+_[0-9]$"))' sample.yml

will output

        this_0
        thisTo_4

  Select items from a map
Given a sample.yml file of:

        things: cat
        bob: goat
        horse: dog

then

        yq '.[] | select(. == "cat" or test("og$"))' sample.yml

will output

        cat
        dog

  Use select and with_entries to filter map keys
Given a sample.yml file of:

        name: bob
        legs: 2
        game: poker

then

        yq 'with_entries(select(.key | test("ame$")))' sample.yml

will output

        name: bob
        game: poker

  Select multiple items in a map and update
Note the brackets around the entire LHS.

Given a sample.yml file of:

        a:
          things: cat
          bob: goat
          horse: dog

then

        yq '(.a.[] | select(. == "cat" or . == "goat")) |= "rabbit"' sample.yml

will output

        a:
          things: rabbit
          bob: rabbit
          horse: dog

Shuffle
Shuffles an array. Note that this command does not use a cryptographically secure random number generator to randomise the array order.

  Shuffle array
Given a sample.yml file of:

        - 1
        - 2
        - 3
        - 4
        - 5

then

        yq 'shuffle' sample.yml

will output

        - 5
        - 2
        - 4
        - 1
        - 3

  Shuffle array in place
Given a sample.yml file of:

        cool:
          - 1
          - 2
          - 3
          - 4
          - 5

then

        yq '.cool |= shuffle' sample.yml

will output

        cool:
          - 5
          - 2
          - 4
          - 1
          - 3

Slice/Splice Array
The slice array operator takes an array as input and returns a subarray. Like the jq equivalent, .[10:15] will return an array of length 5, starting from index 10 inclusive, up to index 15 exclusive. Negative numbers count backwards from the end of the array.

You may leave out the first or second number, which will refer to the start or end of the array respectively.

  Slicing arrays
Given a sample.yml file of:

        - cat
        - dog
        - frog
        - cow

then

        yq '.[1:3]' sample.yml

will output

        - dog
        - frog

  Slicing arrays - without the first number
Starts from the start of the array

Given a sample.yml file of:

        - cat
        - dog
        - frog
        - cow

then

        yq '.[:2]' sample.yml

will output

        - cat
        - dog

  Slicing arrays - without the second number
Finishes at the end of the array

Given a sample.yml file of:

        - cat
        - dog
        - frog
        - cow

then

        yq '.[2:]' sample.yml

will output

        - frog
        - cow

  Slicing arrays - use negative numbers to count backwards from the end
Given a sample.yml file of:

        - cat
        - dog
        - frog
        - cow

then

        yq '.[1:-1]' sample.yml

will output

        - dog
        - frog

  Inserting into the middle of an array
using an expression to find the index

Given a sample.yml file of:

        - cat
        - dog
        - frog
        - cow

then

        yq '(.[] | select(. == "dog") | key + 1) as $pos | .[0:($pos)] + ["rabbit"] + .[$pos:]' sample.yml

will output

        - cat
        - dog
        - rabbit
        - frog
        - cow

Sort Keys
The Sort Keys operator sorts maps by their keys (based on their string value). This operator does not do anything to arrays or scalars (so you can easily recursively apply it to all maps).

Sort is particularly useful for diffing two different yaml documents:

        yq -i -P 'sort_keys(..)' file1.yml
        yq -i -P 'sort_keys(..)' file2.yml
        diff file1.yml file2.yml

Note that yq does not yet consider anchors when sorting by keys - this may result in invalid yaml documents if you are using merge anchors.

For more advanced sorting, using to_entries to convert the map to an array, then sort/process the array as you like (e.g. using sort_by) and convert back to a map using from_entries. See here (https://mikefarah.gitbook.io/yq/operators/entries#custom-sort-map-keys) for an example.

  Sort keys of map
Given a sample.yml file of:

        c: frog
        a: blah
        b: bing

then

        yq 'sort_keys(.)' sample.yml

will output

        a: blah
        b: bing
        c: frog

  Sort keys recursively
Note the array elements are left unsorted, but maps inside arrays are sorted

Given a sample.yml file of:

        bParent:
          c: dog
          array:
            - 3
            - 1
            - 2
        aParent:
          z: donkey
          x:
            - c: yum
              b: delish
            - b: ew
              a: apple

then

        yq 'sort_keys(..)' sample.yml

will output

        aParent:
          x:
            - b: delish
              c: yum
            - a: apple
              b: ew
          z: donkey
        bParent:
          array:
            - 3
            - 1
            - 2
          c: dog

Sort
Sorts an array. Use sort to sort an array as is, or sort_by(exp) to sort by a particular expression (e.g. subfield).

To sort by descending order, pipe the results through the reverse operator after sorting.

Note that at this stage, yq only sorts scalar fields.

  Sort by string field
Given a sample.yml file of:

        - a: banana
        - a: cat
        - a: apple

then

        yq 'sort_by(.a)' sample.yml

will output

        - a: apple
        - a: banana
        - a: cat

  Sort by multiple fields
Given a sample.yml file of:

        - a: dog
        - a: cat
          b: banana
        - a: cat
          b: apple

then

        yq 'sort_by(.a, .b)' sample.yml

will output

        - a: cat
          b: apple
        - a: cat
          b: banana
        - a: dog

  Sort descending by string field
Use sort with reverse to sort in descending order.

Given a sample.yml file of:

        - a: banana
        - a: cat
        - a: apple

then

        yq 'sort_by(.a) | reverse' sample.yml

will output

        - a: cat
        - a: banana
        - a: apple

  Sort array in place
Given a sample.yml file of:

        cool:
          - a: banana
          - a: cat
          - a: apple

then

        yq '.cool |= sort_by(.a)' sample.yml

will output

        cool:
          - a: apple
          - a: banana
          - a: cat

  Sort array of objects by key
Note that you can give sort_by complex expressions, not just paths

Given a sample.yml file of:

        cool:
          - b: banana
          - a: banana
          - c: banana

then

        yq '.cool |= sort_by(keys | .[0])' sample.yml

will output

        cool:
          - a: banana
          - b: banana
          - c: banana

  Sort is stable
Note the order of the elements in unchanged when equal in sorting.

Given a sample.yml file of:

        - a: banana
          b: 1
        - a: banana
          b: 2
        - a: banana
          b: 3
        - a: banana
          b: 4

then

        yq 'sort_by(.a)' sample.yml

will output

        - a: banana
          b: 1
        - a: banana
          b: 2
        - a: banana
          b: 3
        - a: banana
          b: 4

  Sort by numeric field
Given a sample.yml file of:

        - a: 10
        - a: 100
        - a: 1

then

        yq 'sort_by(.a)' sample.yml

will output

        - a: 1
        - a: 10
        - a: 100

  Sort by custom date field
Given a sample.yml file of:

        - a: 12-Jun-2011
        - a: 23-Dec-2010
        - a: 10-Aug-2011

then

        yq 'with_dtf("02-Jan-2006"; sort_by(.a))' sample.yml

will output

        - a: 23-Dec-2010
        - a: 12-Jun-2011
        - a: 10-Aug-2011

  Sort, nulls come first
Given a sample.yml file of:

        - 8
        - 3
        - null
        - 6
        - true
        - false
        - cat

then

        yq 'sort' sample.yml

will output

        - null
        - false
        - true
        - 3
        - 6
        - 8
        - cat

Split into Documents
This operator splits all matches into separate documents

  Split empty
Running

        yq --null-input 'split_doc'

will output

  Split array
Given a sample.yml file of:

        - a: cat
        - b: dog

then

        yq '.[] | split_doc' sample.yml

will output

        a: cat
        ---
        b: dog

String Operators

  RegEx
This uses Golang’s native regex functions under the hood - See their docs (https://github.com/google/re2/wiki/Syntax) for the supported syntax.

Case insensitive tip: prefix the regex with (?i) - e.g. test("(?i)cats)".

  match(regEx)
This operator returns the substring match details of the given regEx.

  capture(regEx)
Capture returns named RegEx capture groups in a map. Can be more convenient than match depending on what you are doing.

  test(regEx)
Returns true if the string matches the RegEx, false otherwise.

  sub(regEx, replacement)
Substitutes matched substrings. The first parameter is the regEx to match substrings within the original string. The second parameter specifies what to replace those matches with. This can refer to capture groups from the first RegEx.

  String blocks, bash and newlines
Bash is notorious for chomping on precious trailing newline characters, making it tricky to set strings with newlines properly. In particular, the $( exp ) will trim trailing newlines.

For instance to get this yaml:

        a: |
          cat

Using $( exp ) wont work, as it will trim the trailing newline.

        m=$(echo "cat\n") yq -n '.a = strenv(m)'
        a: cat

However, using printf works:

        printf -v m "cat\n" ; m="$m" yq -n '.a = strenv(m)'
        a: |
          cat

As well as having multiline expressions:

        m="cat
        "  yq -n '.a = strenv(m)'
        a: |
          cat

Similarly, if you’re trying to set the content from a file, and want a trailing newline:

        IFS= read -rd '' output < <(cat my_file)
        output=$output ./yq '.data.values = strenv(output)' first.yml

  Interpolation
Given a sample.yml file of:

        value: things
        another: stuff

then

        yq '.message = "I like \(.value) and \(.another)"' sample.yml

will output

        value: things
        another: stuff
        message: I like things and stuff

  Interpolation - not a string
Given a sample.yml file of:

        value:
          an: apple

then

        yq '.message = "I like \(.value)"' sample.yml

will output

        value:
          an: apple
        message: 'I like an: apple'

  To up (upper) case
Works with unicode characters

Given a sample.yml file of:

        água

then

        yq 'upcase' sample.yml

will output

        ÁGUA

  To down (lower) case
Works with unicode characters

Given a sample.yml file of:

        ÁgUA

then

        yq 'downcase' sample.yml

will output

        água

  Join strings
Given a sample.yml file of:

        - cat
        - meow
        - 1
        - null
        - true

then

        yq 'join("; ")' sample.yml

will output

        cat; meow; 1; ; true

  Trim strings
Given a sample.yml file of:

        - ' cat'
        - 'dog '
        - ' cow cow '
        - horse

then

        yq '.[] | trim' sample.yml

will output

        cat
        dog
        cow cow
        horse

  Match string
Given a sample.yml file of:

        foo bar foo

then

        yq 'match("foo")' sample.yml

will output

        string: foo
        offset: 0
        length: 3
        captures: []

  Match string, case insensitive
Given a sample.yml file of:

        foo bar FOO

then

        yq '[match("(?i)foo"; "g")]' sample.yml

will output

        - string: foo
          offset: 0
          length: 3
          captures: []
        - string: FOO
          offset: 8
          length: 3
          captures: []

  Match with global capture group
Given a sample.yml file of:

        abc abc

then

        yq '[match("(ab)(c)"; "g")]' sample.yml

will output

        - string: abc
          offset: 0
          length: 3
          captures:
            - string: ab
              offset: 0
              length: 2
            - string: c
              offset: 2
              length: 1
        - string: abc
          offset: 4
          length: 3
          captures:
            - string: ab
              offset: 4
              length: 2
            - string: c
              offset: 6
              length: 1

  Match with named capture groups
Given a sample.yml file of:

        foo bar foo foo  foo

then

        yq '[match("foo (?P<bar123>bar)? foo"; "g")]' sample.yml

will output

        - string: foo bar foo
          offset: 0
          length: 11
          captures:
            - string: bar
              offset: 4
              length: 3
              name: bar123
        - string: foo  foo
          offset: 12
          length: 8
          captures:
            - string: null
              offset: -1
              length: 0
              name: bar123

  Capture named groups into a map
Given a sample.yml file of:

        xyzzy-14

then

        yq 'capture("(?P<a>[a-z]+)-(?P<n>[0-9]+)")' sample.yml

will output

        a: xyzzy
        n: "14"

  Match without global flag
Given a sample.yml file of:

        cat cat

then

        yq 'match("cat")' sample.yml

will output

        string: cat
        offset: 0
        length: 3
        captures: []

  Match with global flag
Given a sample.yml file of:

        cat cat

then

        yq '[match("cat"; "g")]' sample.yml

will output

        - string: cat
          offset: 0
          length: 3
          captures: []
        - string: cat
          offset: 4
          length: 3
          captures: []

  Test using regex
Like jq’s equivalent, this works like match but only returns true/false instead of full match details

Given a sample.yml file of:

        - cat
        - dog

then

        yq '.[] | test("at")' sample.yml

will output

        true
        false

  Substitute / Replace string
This uses Golang’s regex, described here (https://github.com/google/re2/wiki/Syntax) . Note the use of |= to run in context of the current string value.

Given a sample.yml file of:

        a: dogs are great

then

        yq '.a |= sub("dogs", "cats")' sample.yml

will output

        a: cats are great

  Substitute / Replace string with regex
This uses Golang’s regex, described here (https://github.com/google/re2/wiki/Syntax) . Note the use of |= to run in context of the current string value.

Given a sample.yml file of:

        a: cat
        b: heat

then

        yq '.[] |= sub("(a)", "${1}r")' sample.yml

will output

        a: cart
        b: heart

  Custom types: that are really strings
When custom tags are encountered, yq will try to decode the underlying type.

Given a sample.yml file of:

        a: !horse cat
        b: !goat heat

then

        yq '.[] |= sub("(a)", "${1}r")' sample.yml

will output

        a: !horse cart
        b: !goat heart

  Split strings
Given a sample.yml file of:

        cat; meow; 1; ; true

then

        yq 'split("; ")' sample.yml

will output

        - cat
        - meow
        - "1"
        - ""
        - "true"

  Split strings one match
Given a sample.yml file of:

        word

then

        yq 'split("; ")' sample.yml

will output

        - word

  To string
Note that you may want to force yq to leave scalar values wrapped by passing in --unwrapScalar=false or -r=f

Given a sample.yml file of:

        - 1
        - true
        - null
        - ~
        - cat
        - an: object
        - - array
          - 2

then

        yq '.[] |= to_string' sample.yml

will output

        - "1"
        - "true"
        - "null"
        - "~"
        - cat
        - "an: object"
        - "- array\n- 2"

Style
The style operator can be used to get or set the style of nodes (e.g. string style, yaml style). Use this to control the formatting of the document in yaml.

  Update and set style of a particular node (simple)
Given a sample.yml file of:

        a:
          b: thing
          c: something

then

        yq '.a.b = "new" | .a.b style="double"' sample.yml

will output

        a:
          b: "new"
          c: something

  Update and set style of a particular node using path variables
Given a sample.yml file of:

        a:
          b: thing
          c: something

then

        yq 'with(.a.b ; . = "new" | . style="double")' sample.yml

will output

        a:
          b: "new"
          c: something

  Set tagged style
Given a sample.yml file of:

        a: cat
        b: 5
        c: 3.2
        e: true
        f:
          - 1
          - 2
          - 3
        g:
          something: cool

then

        yq '.. style="tagged"' sample.yml

will output

        !!map
        a: !!str cat
        b: !!int 5
        c: !!float 3.2
        e: !!bool true
        f: !!seq
          - !!int 1
          - !!int 2
          - !!int 3
        g: !!map
          something: !!str cool

  Set double quote style
Given a sample.yml file of:

        a: cat
        b: 5
        c: 3.2
        e: true
        f:
          - 1
          - 2
          - 3
        g:
          something: cool

then

        yq '.. style="double"' sample.yml

will output

        a: "cat"
        b: "5"
        c: "3.2"
        e: "true"
        f:
          - "1"
          - "2"
          - "3"
        g:
          something: "cool"

  Set double quote style on map keys too
Given a sample.yml file of:

        a: cat
        b: 5
        c: 3.2
        e: true
        f:
          - 1
          - 2
          - 3
        g:
          something: cool

then

        yq '... style="double"' sample.yml

will output

        "a": "cat"
        "b": "5"
        "c": "3.2"
        "e": "true"
        "f":
          - "1"
          - "2"
          - "3"
        "g":
          "something": "cool"

  Set single quote style
Given a sample.yml file of:

        a: cat
        b: 5
        c: 3.2
        e: true
        f:
          - 1
          - 2
          - 3
        g:
          something: cool

then

        yq '.. style="single"' sample.yml

will output

        a: 'cat'
        b: '5'
        c: '3.2'
        e: 'true'
        f:
          - '1'
          - '2'
          - '3'
        g:
          something: 'cool'

  Set literal quote style
Given a sample.yml file of:

        a: cat
        b: 5
        c: 3.2
        e: true
        f:
          - 1
          - 2
          - 3
        g:
          something: cool

then

        yq '.. style="literal"' sample.yml

will output

        a: |-
          cat
        b: |-
          5
        c: |-
          3.2
        e: |-
          true
        f:
          - |-
            1
          - |-
            2
          - |-
            3
        g:
          something: |-
            cool

  Set folded quote style
Given a sample.yml file of:

        a: cat
        b: 5
        c: 3.2
        e: true
        f:
          - 1
          - 2
          - 3
        g:
          something: cool

then

        yq '.. style="folded"' sample.yml

will output

        a: >-
          cat
        b: >-
          5
        c: >-
          3.2
        e: >-
          true
        f:
          - >-
            1
          - >-
            2
          - >-
            3
        g:
          something: >-
            cool

  Set flow quote style
Given a sample.yml file of:

        a: cat
        b: 5
        c: 3.2
        e: true
        f:
          - 1
          - 2
          - 3
        g:
          something: cool

then

        yq '.. style="flow"' sample.yml

will output

        {a: cat, b: 5, c: 3.2, e: true, f: [1, 2, 3], g: {something: cool}}

  Reset style - or pretty print
Set empty (default) quote style, note the usage of ... to match keys too. Note that there is a --prettyPrint/-P short flag for this.

Given a sample.yml file of:

        {a: cat, "b": 5, 'c': 3.2, "e": true,  f: [1,2,3], "g": { something: "cool"} }

then

        yq '... style=""' sample.yml

will output

        a: cat
        b: 5
        c: 3.2
        e: true
        f:
          - 1
          - 2
          - 3
        g:
          something: cool

  Set style relatively with assign-update
Given a sample.yml file of:

        a: single
        b: double

then

        yq '.[] style |= .' sample.yml

will output

        a: 'single'
        b: "double"

  Read style
Given a sample.yml file of:

        {a: "cat", b: 'thing'}

then

        yq '.. | style' sample.yml

will output

        flow
        double
        single

Subtract
You can use subtract to subtract numbers as well as remove elements from an array.

  Array subtraction
Running

        yq --null-input '[1,2] - [2,3]'

will output

        - 1

  Array subtraction with nested array
Running

        yq --null-input '[[1], 1, 2] - [[1], 3]'

will output

        - 1
        - 2

  Array subtraction with nested object
Note that order of the keys does not matter

Given a sample.yml file of:

        - a: b
          c: d
        - a: b

then

        yq '. - [{"c": "d", "a": "b"}]' sample.yml

will output

        - a: b

  Number subtraction - float
If the lhs or rhs are floats then the expression will be calculated with floats.

Given a sample.yml file of:

        a: 3
        b: 4.5

then

        yq '.a = .a - .b' sample.yml

will output

        a: -1.5
        b: 4.5

  Number subtraction - int
If both the lhs and rhs are ints then the expression will be calculated with ints.

Given a sample.yml file of:

        a: 3
        b: 4

then

        yq '.a = .a - .b' sample.yml

will output

        a: -1
        b: 4

  Decrement numbers
Given a sample.yml file of:

        a: 3
        b: 5

then

        yq '.[] -= 1' sample.yml

will output

        a: 2
        b: 4

  Date subtraction
You can subtract durations from dates. Assumes RFC3339 date time format, see date-time operators (https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information.

Given a sample.yml file of:

        a: 2021-01-01T03:10:00Z

then

        yq '.a -= "3h10m"' sample.yml

will output

        a: 2021-01-01T00:00:00Z

  Date subtraction - custom format
Use with_dtf to specify your datetime format. See date-time operators (https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information.

Given a sample.yml file of:

        a: Saturday, 15-Dec-01 at 6:00AM GMT

then

        yq 'with_dtf("Monday, 02-Jan-06 at 3:04PM MST", .a -= "3h1m")' sample.yml

will output

        a: Saturday, 15-Dec-01 at 2:59AM GMT

  Custom types: that are really numbers
When custom tags are encountered, yq will try to decode the underlying type.

Given a sample.yml file of:

        a: !horse 2
        b: !goat 1

then

        yq '.a -= .b' sample.yml

will output

        a: !horse 1
        b: !goat 1

Tag
The tag operator can be used to get or set the tag of nodes (e.g. !!str, !!int, !!bool).

  Get tag
Given a sample.yml file of:

        a: cat
        b: 5
        c: 3.2
        e: true
        f: []

then

        yq '.. | tag' sample.yml

will output

        !!map
        !!str
        !!int
        !!float
        !!bool
        !!seq

  type is an alias for tag
Given a sample.yml file of:

        a: cat
        b: 5
        c: 3.2
        e: true
        f: []

then

        yq '.. | type' sample.yml

will output

        !!map
        !!str
        !!int
        !!float
        !!bool
        !!seq

  Set custom tag
Given a sample.yml file of:

        a: str

then

        yq '.a tag = "!!mikefarah"' sample.yml

will output

        a: !!mikefarah str

  Find numbers and convert them to strings
Given a sample.yml file of:

        a: cat
        b: 5
        c: 3.2
        e: true

then

        yq '(.. | select(tag == "!!int")) tag= "!!str"' sample.yml

will output

        a: cat
        b: "5"
        c: 3.2
        e: true

To Number
Parses the input as a number. yq will try to parse values as an int first, failing that it will try float. Values that already ints or floats will be left alone.

  Converts strings to numbers
Given a sample.yml file of:

        - "3"
        - "3.1"
        - "-1e3"

then

        yq '.[] | to_number' sample.yml

will output

        3
        3.1
        -1e3

  Doesn’t change numbers
Given a sample.yml file of:

        - 3
        - 3.1
        - -1e3

then

        yq '.[] | to_number' sample.yml

will output

        3
        3.1
        -1e3

  Cannot convert null
Running

        yq --null-input '.a.b | to_number'

will output

        Error: cannot convert node value [null] at path a.b of tag !!null to number

Traverse (Read)
This is the simplest (and perhaps most used) operator. It is used to navigate deeply into yaml structures.

  Simple map navigation
Given a sample.yml file of:

        a:
          b: apple

then

        yq '.a' sample.yml

will output

        b: apple

  Splat
Often used to pipe children into other operators

Given a sample.yml file of:

        - b: apple
        - c: banana

then

        yq '.[]' sample.yml

will output

        b: apple
        c: banana

  Optional Splat
Just like splat, but won’t error if you run it against scalars

Given a sample.yml file of:

        cat

then

        yq '.[]' sample.yml

will output

  Special characters
Use quotes with square brackets around path elements with special characters

Given a sample.yml file of:

        "{}": frog

then

        yq '.["{}"]' sample.yml

will output

        frog

  Nested special characters
Given a sample.yml file of:

        a:
          "key.withdots":
            "another.key": apple

then

        yq '.a["key.withdots"]["another.key"]' sample.yml

will output

        apple

  Keys with spaces
Use quotes with square brackets around path elements with special characters

Given a sample.yml file of:

        "red rabbit": frog

then

        yq '.["red rabbit"]' sample.yml

will output

        frog

  Dynamic keys
Expressions within [] can be used to dynamically lookup / calculate keys

Given a sample.yml file of:

        b: apple
        apple: crispy yum
        banana: soft yum

then

        yq '.[.b]' sample.yml

will output

        crispy yum

  Children don’t exist
Nodes are added dynamically while traversing

Given a sample.yml file of:

        c: banana

then

        yq '.a.b' sample.yml

will output

        null

  Optional identifier
Like jq, does not output an error when the yaml is not an array or object as expected

Given a sample.yml file of:

        - 1
        - 2
        - 3

then

        yq '.a?' sample.yml

will output

  Wildcard matching
Given a sample.yml file of:

        a:
          cat: apple
          mad: things

then

        yq '.a."*a*"' sample.yml

will output

        apple
        things

  Aliases
Given a sample.yml file of:

        a: &cat
          c: frog
        b: *cat

then

        yq '.b' sample.yml

will output

        *cat

  Traversing aliases with splat
Given a sample.yml file of:

        a: &cat
          c: frog
        b: *cat

then

        yq '.b[]' sample.yml

will output

        frog

  Traversing aliases explicitly
Given a sample.yml file of:

        a: &cat
          c: frog
        b: *cat

then

        yq '.b.c' sample.yml

will output

        frog

  Traversing arrays by index
Given a sample.yml file of:

        - 1
        - 2
        - 3

then

        yq '.[0]' sample.yml

will output

        1

  Traversing nested arrays by index
Given a sample.yml file of:

        [[], [cat]]

then

        yq '.[1][0]' sample.yml

will output

        cat

  Maps with numeric keys
Given a sample.yml file of:

        2: cat

then

        yq '.[2]' sample.yml

will output

        cat

  Maps with non existing numeric keys
Given a sample.yml file of:

        a: b

then

        yq '.[0]' sample.yml

will output

        null

  Traversing merge anchors
Given a sample.yml file of:

        foo: &foo
          a: foo_a
          thing: foo_thing
          c: foo_c
        bar: &bar
          b: bar_b
          thing: bar_thing
          c: bar_c
        foobarList:
          b: foobarList_b
          !!merge <<:
            - *foo
            - *bar
          c: foobarList_c
        foobar:
          c: foobar_c
          !!merge <<: *foo
          thing: foobar_thing

then

        yq '.foobar.a' sample.yml

will output

        foo_a

  Traversing merge anchors with override
Given a sample.yml file of:

        foo: &foo
          a: foo_a
          thing: foo_thing
          c: foo_c
        bar: &bar
          b: bar_b
          thing: bar_thing
          c: bar_c
        foobarList:
          b: foobarList_b
          !!merge <<:
            - *foo
            - *bar
          c: foobarList_c
        foobar:
          c: foobar_c
          !!merge <<: *foo
          thing: foobar_thing

then

        yq '.foobar.c' sample.yml

will output

        foo_c

  Traversing merge anchors with local override
Given a sample.yml file of:

        foo: &foo
          a: foo_a
          thing: foo_thing
          c: foo_c
        bar: &bar
          b: bar_b
          thing: bar_thing
          c: bar_c
        foobarList:
          b: foobarList_b
          !!merge <<:
            - *foo
            - *bar
          c: foobarList_c
        foobar:
          c: foobar_c
          !!merge <<: *foo
          thing: foobar_thing

then

        yq '.foobar.thing' sample.yml

will output

        foobar_thing

  Splatting merge anchors
Given a sample.yml file of:

        foo: &foo
          a: foo_a
          thing: foo_thing
          c: foo_c
        bar: &bar
          b: bar_b
          thing: bar_thing
          c: bar_c
        foobarList:
          b: foobarList_b
          !!merge <<:
            - *foo
            - *bar
          c: foobarList_c
        foobar:
          c: foobar_c
          !!merge <<: *foo
          thing: foobar_thing

then

        yq '.foobar[]' sample.yml

will output

        foo_c
        foo_a
        foobar_thing

  Traversing merge anchor lists
Note that the later merge anchors override previous

Given a sample.yml file of:

        foo: &foo
          a: foo_a
          thing: foo_thing
          c: foo_c
        bar: &bar
          b: bar_b
          thing: bar_thing
          c: bar_c
        foobarList:
          b: foobarList_b
          !!merge <<:
            - *foo
            - *bar
          c: foobarList_c
        foobar:
          c: foobar_c
          !!merge <<: *foo
          thing: foobar_thing

then

        yq '.foobarList.thing' sample.yml

will output

        bar_thing

  Splatting merge anchor lists
Given a sample.yml file of:

        foo: &foo
          a: foo_a
          thing: foo_thing
          c: foo_c
        bar: &bar
          b: bar_b
          thing: bar_thing
          c: bar_c
        foobarList:
          b: foobarList_b
          !!merge <<:
            - *foo
            - *bar
          c: foobarList_c
        foobar:
          c: foobar_c
          !!merge <<: *foo
          thing: foobar_thing

then

        yq '.foobarList[]' sample.yml

will output

        bar_b
        foo_a
        bar_thing
        foobarList_c

  Select multiple indices
Given a sample.yml file of:

        a:
          - a
          - b
          - c

then

        yq '.a[0, 2]' sample.yml

will output

        a
        c

Union
This operator is used to combine different results together.

  Combine scalars
Running

        yq --null-input '1, true, "cat"'

will output

        1
        true
        cat

  Combine selected paths
Given a sample.yml file of:

        a: fieldA
        b: fieldB
        c: fieldC

then

        yq '.a, .c' sample.yml

will output

        fieldA
        fieldC

Unique
This is used to filter out duplicated items in an array. Note that the original order of the array is maintained.

  Unique array of scalars (string/numbers)
Note that unique maintains the original order of the array.

Given a sample.yml file of:

        - 2
        - 1
        - 3
        - 2

then

        yq 'unique' sample.yml

will output

        - 2
        - 1
        - 3

  Unique nulls
Unique works on the node value, so it considers different representations of nulls to be different

Given a sample.yml file of:

        - ~
        - null
        - ~
        - null

then

        yq 'unique' sample.yml

will output

        - ~
        - null

  Unique all nulls
Run against the node tag to unique all the nulls

Given a sample.yml file of:

        - ~
        - null
        - ~
        - null

then

        yq 'unique_by(tag)' sample.yml

will output

        - ~

  Unique array object fields
Given a sample.yml file of:

        - name: harry
          pet: cat
        - name: billy
          pet: dog
        - name: harry
          pet: dog

then

        yq 'unique_by(.name)' sample.yml

will output

        - name: harry
          pet: cat
        - name: billy
          pet: dog

Variable Operators
Like the jq equivalents, variables are sometimes required for the more complex expressions (or swapping values between fields).

Note that there is also an additional ref operator that holds a reference (instead of a copy) of the path, allowing you to make multiple changes to the same path.

  Single value variable
Given a sample.yml file of:

        a: cat

then

        yq '.a as $foo | $foo' sample.yml

will output

        cat

  Multi value variable
Given a sample.yml file of:

        - cat
        - dog

then

        yq '.[] as $foo | $foo' sample.yml

will output

        cat
        dog

  Using variables as a lookup
Example taken from jq (https://stedolan.github.io/jq/manual/#Variable/SymbolicBindingOperator:...as$identifier%7C...)

Given a sample.yml file of:

        "posts":
          - "title": First post
            "author": anon
          - "title": A well-written article
            "author": person1
        "realnames":
          "anon": Anonymous Coward
          "person1": Person McPherson

then

        yq '.realnames as $names | .posts[] | {"title":.title, "author": $names[.author]}' sample.yml

will output

        title: First post
        author: Anonymous Coward
        title: A well-written article
        author: Person McPherson

  Using variables to swap values
Given a sample.yml file of:

        a: a_value
        b: b_value

then

        yq '.a as $x  | .b as $y | .b = $x | .a = $y' sample.yml

will output

        a: b_value
        b: a_value

  Use ref to reference a path repeatedly
Note: You may find the with operator more useful.

Given a sample.yml file of:

        a:
          b: thing
          c: something

then

        yq '.a.b ref $x | $x = "new" | $x style="double"' sample.yml

will output

        a:
          b: "new"
          c: something

With
Use the with operator to conveniently make multiple updates to a deeply nested path, or to update array elements relatively to each other. The first argument expression sets the root context, and the second expression runs against that root context.

  Update and style
Given a sample.yml file of:

        a:
          deeply:
            nested: value

then

        yq 'with(.a.deeply.nested; . = "newValue" | . style="single")' sample.yml

will output

        a:
          deeply:
            nested: 'newValue'

  Update multiple deeply nested properties
Given a sample.yml file of:

        a:
          deeply:
            nested: value
            other: thing

then

        yq 'with(.a.deeply; .nested = "newValue" | .other= "newThing")' sample.yml

will output

        a:
          deeply:
            nested: newValue
            other: newThing

  Update array elements relatively
The second expression runs with each element of the array as it’s contextual root. This allows you to make updates relative to the element.

Given a sample.yml file of:

        myArray:
          - a: apple
          - a: banana

then

        yq 'with(.myArray[]; .b = .a + " yum")' sample.yml

will output

        myArray:
          - a: apple
            b: apple yum
          - a: banana
            b: banana yum

JSON
Encode and decode to and from JSON. Supports multiple JSON documents in a single file (e.g. NDJSON).

Note that YAML is a superset of (single document) JSON - so you don’t have to use the JSON parser to read JSON when there is only one JSON document in the input. You will probably want to pretty print the result in this case, to get idiomatic YAML styling.

  Parse json: simple
JSON is a subset of yaml, so all you need to do is prettify the output

Given a sample.json file of:

        {"cat": "meow"}

then

        yq -p=json sample.json

will output

        cat: meow

  Parse json: complex
JSON is a subset of yaml, so all you need to do is prettify the output

Given a sample.json file of:

        {"a":"Easy! as one two three","b":{"c":2,"d":[3,4]}}

then

        yq -p=json sample.json

will output

        a: Easy! as one two three
        b:
          c: 2
          d:
            - 3
            - 4

  Encode json: simple
Given a sample.yml file of:

        cat: meow

then

        yq -o=json '.' sample.yml

will output

        {
          "cat": "meow"
        }

  Encode json: simple - in one line
Given a sample.yml file of:

        cat: meow # this is a comment, and it will be dropped.

then

        yq -o=json -I=0 '.' sample.yml

will output

        {"cat":"meow"}

  Encode json: comments
Given a sample.yml file of:

        cat: meow # this is a comment, and it will be dropped.

then

        yq -o=json '.' sample.yml

will output

        {
          "cat": "meow"
        }

  Encode json: anchors
Anchors are dereferenced

Given a sample.yml file of:

        cat: &ref meow
        anotherCat: *ref

then

        yq -o=json '.' sample.yml

will output

        {
          "cat": "meow",
          "anotherCat": "meow"
        }

  Encode json: multiple results
Each matching node is converted into a json doc. This is best used with 0 indent (json document per line)

Given a sample.yml file of:

        things: [{stuff: cool}, {whatever: cat}]

then

        yq -o=json -I=0 '.things[]' sample.yml

will output

        {"stuff":"cool"}
        {"whatever":"cat"}

  Roundtrip JSON Lines / NDJSON
Given a sample.json file of:

        {"this": "is a multidoc json file"}
        {"each": ["line is a valid json document"]}
        {"a number": 4}

then

        yq -p=json -o=json -I=0 sample.json

will output

        {"this":"is a multidoc json file"}
        {"each":["line is a valid json document"]}
        {"a number":4}

  Roundtrip multi-document JSON
The parser can also handle multiple multi-line json documents in a single file (despite this not being in the JSON Lines / NDJSON spec). Typically you would have one entire JSON document per line, but the parser also supports multiple multi-line json documents

Given a sample.json file of:

        {
            "this": "is a multidoc json file"
        }
        {
            "it": [
                "has",
                "consecutive",
                "json documents"
            ]
        }
        {
            "a number": 4
        }

then

        yq -p=json -o=json -I=2 sample.json

will output

        {
          "this": "is a multidoc json file"
        }
        {
          "it": [
            "has",
            "consecutive",
            "json documents"
          ]
        }
        {
          "a number": 4
        }

  Update a specific document in a multi-document json
Documents are indexed by the documentIndex or di operator.

Given a sample.json file of:

        {"this": "is a multidoc json file"}
        {"each": ["line is a valid json document"]}
        {"a number": 4}

then

        yq -p=json -o=json -I=0 '(select(di == 1) | .each ) += "cool"' sample.json

will output

        {"this":"is a multidoc json file"}
        {"each":["line is a valid json document","cool"]}
        {"a number":4}

  Find and update a specific document in a multi-document json
Use expressions as you normally would.

Given a sample.json file of:

        {"this": "is a multidoc json file"}
        {"each": ["line is a valid json document"]}
        {"a number": 4}

then

        yq -p=json -o=json -I=0 '(select(has("each")) | .each ) += "cool"' sample.json

will output

        {"this":"is a multidoc json file"}
        {"each":["line is a valid json document","cool"]}
        {"a number":4}

  Decode JSON Lines / NDJSON
Given a sample.json file of:

        {"this": "is a multidoc json file"}
        {"each": ["line is a valid json document"]}
        {"a number": 4}

then

        yq -p=json sample.json

will output

        this: is a multidoc json file
        ---
        each:
          - line is a valid json document
        ---
        a number: 4

CSV
Encode/Decode/Roundtrip CSV and TSV files.

  Encode
Currently supports arrays of homogeneous flat objects, that is: no nesting and it assumes the first object has all the keys required:

        - name: Bobo
          type: dog
        - name: Fifi
          type: cat

As well as arrays of arrays of scalars (strings/numbers/booleans):

        - [Bobo, dog]
        - [Fifi, cat]

  Decode
Decode assumes the first CSV/TSV row is the header row, and all rows beneath are the entries. The data will be coded into an array of objects, using the header rows as keys.

        name,type
        Bobo,dog
        Fifi,cat

  Encode CSV simple
Given a sample.yml file of:

        - [i, like, csv]
        - [because, excel, is, cool]

then

        yq -o=csv sample.yml

will output

        i,like,csv
        because,excel,is,cool

  Encode TSV simple
Given a sample.yml file of:

        - [i, like, csv]
        - [because, excel, is, cool]

then

        yq -o=tsv sample.yml

will output

        i   like    csv
        because excel   is  cool

  Encode array of objects to csv
Given a sample.yml file of:

        - name: Gary
          numberOfCats: 1
          likesApples: true
          height: 168.8
        - name: Samantha's Rabbit
          numberOfCats: 2
          likesApples: false
          height: -188.8

then

        yq -o=csv sample.yml

will output

        name,numberOfCats,likesApples,height
        Gary,1,true,168.8
        Samantha's Rabbit,2,false,-188.8

  Encode array of objects to custom csv format
Add the header row manually, then the we convert each object into an array of values - resulting in an array of arrays. Pick the columns and call the header whatever you like.

Given a sample.yml file of:

        - name: Gary
          numberOfCats: 1
          likesApples: true
          height: 168.8
        - name: Samantha's Rabbit
          numberOfCats: 2
          likesApples: false
          height: -188.8

then

        yq -o=csv '[["Name", "Number of Cats"]] +  [.[] | [.name, .numberOfCats ]]' sample.yml

will output

        Name,Number of Cats
        Gary,1
        Samantha's Rabbit,2

  Encode array of objects to csv - missing fields behaviour
First entry is used to determine the headers, and it is missing `likesApples', so it is not included in the csv. Second entry does not have `numberOfCats' so that is blank

Given a sample.yml file of:

        - name: Gary
          numberOfCats: 1
          height: 168.8
        - name: Samantha's Rabbit
          height: -188.8
          likesApples: false

then

        yq -o=csv sample.yml

will output

        name,numberOfCats,height
        Gary,1,168.8
        Samantha's Rabbit,,-188.8

  Parse CSV into an array of objects
First row is assumed to be the header row. By default, entries with YAML/JSON formatting will be parsed!

Given a sample.csv file of:

        name,numberOfCats,likesApples,height,facts
        Gary,1,true,168.8,cool: true
        Samantha's Rabbit,2,false,-188.8,tall: indeed

then

        yq -p=csv sample.csv

will output

        - name: Gary
          numberOfCats: 1
          likesApples: true
          height: 168.8
          facts:
            cool: true
        - name: Samantha's Rabbit
          numberOfCats: 2
          likesApples: false
          height: -188.8
          facts:
            tall: indeed

  Parse CSV into an array of objects, no auto-parsing
First row is assumed to be the header row. Entries with YAML/JSON will be left as strings.

Given a sample.csv file of:

        name,numberOfCats,likesApples,height,facts
        Gary,1,true,168.8,cool: true
        Samantha's Rabbit,2,false,-188.8,tall: indeed

then

        yq -p=csv --csv-auto-parse=f sample.csv

will output

        - name: Gary
          numberOfCats: 1
          likesApples: true
          height: 168.8
          facts: 'cool: true'
        - name: Samantha's Rabbit
          numberOfCats: 2
          likesApples: false
          height: -188.8
          facts: 'tall: indeed'

  Parse TSV into an array of objects
First row is assumed to be the header row.

Given a sample.tsv file of:

        name    numberOfCats    likesApples height
        Gary    1   true    168.8
        Samantha's Rabbit   2   false   -188.8

then

        yq -p=tsv sample.tsv

will output

        - name: Gary
          numberOfCats: 1
          likesApples: true
          height: 168.8
        - name: Samantha's Rabbit
          numberOfCats: 2
          likesApples: false
          height: -188.8

  Round trip
Given a sample.csv file of:

        name,numberOfCats,likesApples,height
        Gary,1,true,168.8
        Samantha's Rabbit,2,false,-188.8

then

        yq -p=csv -o=csv '(.[] | select(.name == "Gary") | .numberOfCats) = 3' sample.csv

will output

        name,numberOfCats,likesApples,height
        Gary,3,true,168.8
        Samantha's Rabbit,2,false,-188.8

Formatting Expressions
From version v4.41+

You can put expressions into .yq files, use whitespace and comments to break up complex expressions and explain what’s going on.

  Using expression files and comments
Note that you can execute the file directly - but make sure you make the expression file executable.

Given a sample.yaml file of:

        a:
          b: old

And an `update.yq' expression file of:

        #! yq
        # This is a yq expression that updates the map
        # for several great reasons outlined here.
        .a.b = "new" # line comment here
        | .a.c = "frog"
        # Now good things will happen.

then

        ./update.yq sample.yaml

will output

        a:
          b: new
          c: frog

  Flags in expression files
You can specify flags on the shebang line, this only works when executing the file directly.

Given a sample.yaml file of:

        a:
          b: old

And an `update.yq' expression file of:

        #! yq -oj
        # This is a yq expression that updates the map
        # for several great reasons outlined here.
        .a.b = "new" # line comment here
        | .a.c = "frog"
        # Now good things will happen.

then

        ./update.yq sample.yaml

will output

        {
          "a": {
            "b": "new",
            "c": "frog"
          }
        }

  Commenting out yq expressions
Note that c is no longer set to `frog'. In this example we’re calling yq directly and passing the expression file into --from-file, this is no different from executing the expression file directly.

Given a sample.yaml file of:

        a:
          b: old

And an `update.yq' expression file of:

        #! yq
        # This is a yq expression that updates the map
        # for several great reasons outlined here.
        .a.b = "new" # line comment here
        # | .a.c = "frog"
        # Now good things will happen.

then

        yq --from-file update.yq sample.yml

will output

        a:
          b: new

  Basic input example
Given a sample.lua file of:

        return {
            ["country"] = "Australia"; -- this place
            ["cities"] = {
                "Sydney",
                "Melbourne",
                "Brisbane",
                "Perth",
            };
        };

then

        yq -oy '.' sample.lua

will output

        country: Australia
        cities:
          - Sydney
          - Melbourne
          - Brisbane
          - Perth

  Basic output example
Given a sample.yml file of:

        ---
        country: Australia # this place
        cities:
        - Sydney
        - Melbourne
        - Brisbane
        - Perth

then

        yq -o=lua '.' sample.yml

will output

        return {
            ["country"] = "Australia"; -- this place
            ["cities"] = {
                "Sydney",
                "Melbourne",
                "Brisbane",
                "Perth",
            };
        };

  Unquoted keys
Uses the --lua-unquoted option to produce a nicer-looking output.

Given a sample.yml file of:

        ---
        country: Australia # this place
        cities:
        - Sydney
        - Melbourne
        - Brisbane
        - Perth

then

        yq -o=lua --lua-unquoted '.' sample.yml

will output

        return {
            country = "Australia"; -- this place
            cities = {
                "Sydney",
                "Melbourne",
                "Brisbane",
                "Perth",
            };
        };

  Globals
Uses the --lua-globals option to export the values into the global scope.

Given a sample.yml file of:

        ---
        country: Australia # this place
        cities:
        - Sydney
        - Melbourne
        - Brisbane
        - Perth

then

        yq -o=lua --lua-globals '.' sample.yml

will output

        country = "Australia"; -- this place
        cities = {
            "Sydney",
            "Melbourne",
            "Brisbane",
            "Perth",
        };

  Elaborate example
Given a sample.yml file of:

        ---
        hello: world
        tables:
          like: this
          keys: values
          ? look: non-string keys
          : True
        numbers:
          - decimal: 12345
          - hex: 0x7fabc123
          - octal: 0o30
          - float: 123.45
          - infinity: .inf
            plus_infinity: +.inf
            minus_infinity: -.inf
          - not: .nan

then

        yq -o=lua '.' sample.yml

will output

        return {
            ["hello"] = "world";
            ["tables"] = {
                ["like"] = "this";
                ["keys"] = "values";
                [{
                    ["look"] = "non-string keys";
                }] = true;
            };
            ["numbers"] = {
                {
                    ["decimal"] = 12345;
                },
                {
                    ["hex"] = 0x7fabc123;
                },
                {
                    ["octal"] = 24;
                },
                {
                    ["float"] = 123.45;
                },
                {
                    ["infinity"] = (1/0);
                    ["plus_infinity"] = (1/0);
                    ["minus_infinity"] = (-1/0);
                },
                {
                    ["not"] = (0/0);
                },
            };
        };

Properties
Encode/Decode/Roundtrip to/from a property file. Line comments on value nodes will be copied across.

By default, empty maps and arrays are not encoded - see below for an example on how to encode a value for these.

  Encode properties
Note that empty arrays and maps are not encoded by default.

Given a sample.yml file of:

        # block comments come through
        person: # neither do comments on maps
            name: Mike Wazowski # comments on values appear
            pets:
            - cat # comments on array values appear
            - nested:
                - list entry
            food: [pizza] # comments on arrays do not
        emptyArray: []
        emptyMap: []

then

        yq -o=props sample.yml

will output

        # block comments come through
        # comments on values appear
        person.name = Mike Wazowski
        # comments on array values appear
        person.pets.0 = cat
        person.pets.1.nested.0 = list entry
        person.food.0 = pizza

  Encode properties with array brackets
Declare the –properties-array-brackets flag to give array paths in brackets (e.g. SpringBoot).

Given a sample.yml file of:

        # block comments come through
        person: # neither do comments on maps
            name: Mike Wazowski # comments on values appear
            pets:
            - cat # comments on array values appear
            - nested:
                - list entry
            food: [pizza] # comments on arrays do not
        emptyArray: []
        emptyMap: []

then

        yq -o=props --properties-array-brackets sample.yml

will output

        # block comments come through
        # comments on values appear
        person.name = Mike Wazowski
        # comments on array values appear
        person.pets[0] = cat
        person.pets[1].nested[0] = list entry
        person.food[0] = pizza

  Encode properties - custom separator
Use the –properties-customer-separator flag to specify your own key/value separator.

Given a sample.yml file of:

        # block comments come through
        person: # neither do comments on maps
            name: Mike Wazowski # comments on values appear
            pets:
            - cat # comments on array values appear
            - nested:
                - list entry
            food: [pizza] # comments on arrays do not
        emptyArray: []
        emptyMap: []

then

        yq -o=props --properties-customer-separator=" :@ " sample.yml

will output

        # block comments come through
        # comments on values appear
        person.name :@ Mike Wazowski
        # comments on array values appear
        person.pets.0 :@ cat
        person.pets.1.nested.0 :@ list entry
        person.food.0 :@ pizza

  Encode properties: scalar encapsulation
Note that string values with blank characters in them are encapsulated with double quotes

Given a sample.yml file of:

        # block comments come through
        person: # neither do comments on maps
            name: Mike Wazowski # comments on values appear
            pets:
            - cat # comments on array values appear
            - nested:
                - list entry
            food: [pizza] # comments on arrays do not
        emptyArray: []
        emptyMap: []

then

        yq -o=props --unwrapScalar=false sample.yml

will output

        # block comments come through
        # comments on values appear
        person.name = "Mike Wazowski"
        # comments on array values appear
        person.pets.0 = cat
        person.pets.1.nested.0 = "list entry"
        person.food.0 = pizza

  Encode properties: no comments
Given a sample.yml file of:

        # block comments come through
        person: # neither do comments on maps
            name: Mike Wazowski # comments on values appear
            pets:
            - cat # comments on array values appear
            - nested:
                - list entry
            food: [pizza] # comments on arrays do not
        emptyArray: []
        emptyMap: []

then

        yq -o=props '... comments = ""' sample.yml

will output

        person.name = Mike Wazowski
        person.pets.0 = cat
        person.pets.1.nested.0 = list entry
        person.food.0 = pizza

  Encode properties: include empty maps and arrays
Use a yq expression to set the empty maps and sequences to your desired value.

Given a sample.yml file of:

        # block comments come through
        person: # neither do comments on maps
            name: Mike Wazowski # comments on values appear
            pets:
            - cat # comments on array values appear
            - nested:
                - list entry
            food: [pizza] # comments on arrays do not
        emptyArray: []
        emptyMap: []

then

        yq -o=props '(.. | select( (tag == "!!map" or tag =="!!seq") and length == 0)) = ""' sample.yml

will output

        # block comments come through
        # comments on values appear
        person.name = Mike Wazowski
        # comments on array values appear
        person.pets.0 = cat
        person.pets.1.nested.0 = list entry
        person.food.0 = pizza
        emptyArray =
        emptyMap =

  Decode properties
Given a sample.properties file of:

        # block comments come through
        # comments on values appear
        person.name = Mike Wazowski
        # comments on array values appear
        person.pets.0 = cat
        person.pets.1.nested.0 = list entry
        person.food.0 = pizza

then

        yq -p=props sample.properties

will output

        person:
          # block comments come through
          # comments on values appear
          name: Mike Wazowski
          pets:
            # comments on array values appear
            - cat
            - nested:
                - list entry
          food:
            - pizza

  Decode properties: numbers
All values are assumed to be strings when parsing properties, but you can use the from_yaml operator on all the strings values to autoparse into the correct type.

Given a sample.properties file of:

        a.b = 10

then

        yq -p=props ' (.. | select(tag == "!!str")) |= from_yaml' sample.properties

will output

        a:
          b: 10

  Decode properties - array should be a map
If you have a numeric map key in your property files, use array_to_map to convert them to maps.

Given a sample.properties file of:

        things.10 = mike

then

        yq -p=props '.things |= array_to_map' sample.properties

will output

        things:
          10: mike

  Roundtrip
Given a sample.properties file of:

        # block comments come through
        # comments on values appear
        person.name = Mike Wazowski
        # comments on array values appear
        person.pets.0 = cat
        person.pets.1.nested.0 = list entry
        person.food.0 = pizza

then

        yq -p=props -o=props '.person.pets.0 = "dog"' sample.properties

will output

        # block comments come through
        # comments on values appear
        person.name = Mike Wazowski
        # comments on array values appear
        person.pets.0 = dog
        person.pets.1.nested.0 = list entry
        person.food.0 = pizza

Recipes
These examples are intended to show how you can use multiple operators together so you get an idea of how you can perform complex data manipulation.

Please see the details operator docs (https://mikefarah.gitbook.io/yq/operators) for details on each individual operator.

  Find items in an array
We have an array and we want to find the elements with a particular name.

Given a sample.yml file of:

        - name: Foo
          numBuckets: 0
        - name: Bar
          numBuckets: 0

then

        yq '.[] | select(.name == "Foo")' sample.yml

will output

        name: Foo
        numBuckets: 0

  Explanation:

•   .[] splats the array, and puts all the items in the context.

•   These items are then piped (|) into select(.name == "Foo") which will select all the nodes that have a name property set to `Foo'.

•   See the select (https://mikefarah.gitbook.io/yq/operators/select) operator for more information.

  Find and update items in an array
We have an array and we want to update the elements with a particular name.

Given a sample.yml file of:

        - name: Foo
          numBuckets: 0
        - name: Bar
          numBuckets: 0

then

        yq '(.[] | select(.name == "Foo") | .numBuckets) |= . + 1' sample.yml

will output

        - name: Foo
          numBuckets: 1
        - name: Bar
          numBuckets: 0

  Explanation:

•   Following from the example above.[] splats the array, selects filters the items.

•   We then pipe (|) that into .numBuckets, which will select that field from all the matching items

•   Splat, select and the field are all in brackets, that whole expression is passed to the |= operator as the left hand side expression, with . + 1 as the right hand side expression.

•   |= is the operator that updates fields relative to their own value, which is referenced as dot (.).

•   The expression . + 1 increments the numBuckets counter.

•   See the assign (https://mikefarah.gitbook.io/yq/operators/assign-update) and add (https://mikefarah.gitbook.io/yq/operators/add) operators for more information.

  Deeply prune a tree
Say we are only interested in child1 and child2, and want to filter everything else out.

Given a sample.yml file of:

        parentA:
          - bob
        parentB:
          child1: i am child1
          child3: hiya
        parentC:
          childX: cool
          child2: me child2

then

        yq '(
          .. | # recurse through all the nodes
          select(has("child1") or has("child2")) | # match parents that have either child1 or child2
          (.child1, .child2) | # select those children
          select(.) # filter out nulls
        ) as $i ireduce({};  # using that set of nodes, create a new result map
          setpath($i | path; $i) # and put in each node, using its original path
        )' sample.yml

will output

        parentB:
          child1: i am child1
        parentC:
          child2: me child2

  Explanation:

•   Find all the matching child1 and child2 nodes

•   Using ireduce, create a new map using just those nodes

•   Set each node into the new map using its original path

  Multiple or complex updates to items in an array
We have an array and we want to update the elements with a particular name in reference to its type.

Given a sample.yml file of:

        myArray:
          - name: Foo
            type: cat
          - name: Bar
            type: dog

then

        yq 'with(.myArray[]; .name = .name + " - " + .type)' sample.yml

will output

        myArray:
          - name: Foo - cat
            type: cat
          - name: Bar - dog
            type: dog

  Explanation:

•   The with operator will effectively loop through each given item in the first given expression, and run the second expression against it.

•   .myArray[] splats the array in myArray. So with will run against each item in that array

•   .name = .name + " - " + .type this expression is run against every item, updating the name to be a concatenation of the original name as well as the type.

•   See the with (https://mikefarah.gitbook.io/yq/operators/with) operator for more information and examples.

  Sort an array by a field
Given a sample.yml file of:

        myArray:
          - name: Foo
            numBuckets: 1
          - name: Bar
            numBuckets: 0

then

        yq '.myArray |= sort_by(.numBuckets)' sample.yml

will output

        myArray:
          - name: Bar
            numBuckets: 0
          - name: Foo
            numBuckets: 1

  Explanation:

•   We want to resort .myArray.

•   sort_by works by piping an array into it, and it pipes out a sorted array.

•   So, we use |= to update .myArray. This is the same as doing .myArray = (.myArray | sort_by(.numBuckets))

  Filter, flatten, sort and unique
Lets find the unique set of names from the document.

Given a sample.yml file of:

        - type: foo
          names:
            - Fred
            - Catherine
        - type: bar
          names:
            - Zelda
        - type: foo
          names: Fred
        - type: foo
          names: Ava

then

        yq '[.[] | select(.type == "foo") | .names] | flatten | sort | unique' sample.yml

will output

        - Ava
        - Catherine
        - Fred

  Explanation:

•   .[] | select(.type == "foo") | .names will select the array elements of type “foo”

•   Splat .[] will unwrap the array and match all the items. We need to do this so we can work on the child items, for instance, filter items out using the select operator.

•   But we still want the final results back into an array. So after we’re doing working on the children, we wrap everything back into an array using square brackets around the expression. [.[] | select(.type == "foo") | .names]

•   Now have have an array of all the `names' values. Which includes arrays of strings as well as strings on their own.

•   Pipe | this array through flatten. This will flatten nested arrays. So now we have a flat list of all the name value strings

•   Next we pipe | that through sort and then unique to get a sorted, unique list of the names!

•   See the flatten (https://mikefarah.gitbook.io/yq/operators/flatten) , sort (https://mikefarah.gitbook.io/yq/operators/sort) and unique (https://mikefarah.gitbook.io/yq/operators/unique) for more information and examples.

  Export as environment variables (script), or any custom format
Given a yaml document, lets output a script that will configure environment variables with that data. This same approach can be used for exporting into custom formats.

Given a sample.yml file of:

        var0: string0
        var1: string1
        fruit:
          - apple
          - banana
          - peach

then

        yq '.[] |(
            ( select(kind == "scalar") | key + "='\''" + . + "'\''"),
            ( select(kind == "seq") | key + "=(" + (map("'\''" + . + "'\''") | join(",")) + ")")
        )' sample.yml

will output

        var0='string0'
        var1='string1'
        fruit=('apple','banana','peach')

  Explanation:

•   .[] matches all top level elements

•   We need a string expression for each of the different types that will produce the bash syntax, we’ll use the union operator, to join them together

•   Scalars, we just need the key and quoted value: ( select(kind == "scalar") | key + "='" + . + "'")

•   Sequences (or arrays) are trickier, we need to quote each value and join them with ,: map("'" + . + "'") | join(",")

  Custom format with nested data
Like the previous example, but lets handle nested data structures. In this custom example, we’re going to join the property paths with _. The important thing to keep in mind is that our expression is not recursive (despite the data structure being so). Instead we match all elements on the tree and operate on them.

Given a sample.yml file of:

        simple: string0
        simpleArray:
          - apple
          - banana
          - peach
        deep:
          property: value
          array:
            - cat

then

        yq '.. |(
            ( select(kind == "scalar" and parent | kind != "seq") | (path | join("_")) + "='\''" + . + "'\''"),
            ( select(kind == "seq") | (path | join("_")) + "=(" + (map("'\''" + . + "'\''") | join(",")) + ")")
        )' sample.yml

will output

        simple='string0'
        deep_property='value'
        simpleArray=('apple','banana','peach')
        deep_array=('cat')

  Explanation:

•   You’ll need to understand how the previous example works to understand this extension.

•   .. matches all elements, instead of .[] from the previous example that just matches top level elements.

•   Like before, we need a string expression for each of the different types that will produce the bash syntax, we’ll use the union operator, to join them together

•   This time, however, our expression matches every node in the data structure.

•   We only want to print scalars that are not in arrays (because we handle the separately), so well add and parent | kind != "seq" to the select operator expression for scalars

•   We don’t just want the key any more, we want the full path. So instead of key we have path | join("_")

•   The expression for sequences follows the same logic

  Encode shell variables
Note that comments are dropped and values will be enclosed in single quotes as needed.

Given a sample.yml file of:

        # comment
        name: Mike Wazowski
        eyes:
          color: turquoise
          number: 1
        friends:
          - James P. Sullivan
          - Celia Mae

then

        yq -o=shell sample.yml

will output

        name='Mike Wazowski'
        eyes_color=turquoise
        eyes_number=1
        friends_0='James P. Sullivan'
        friends_1='Celia Mae'

  Encode shell variables: illegal variable names as key.
Keys that would be illegal as variable keys are adapted.

Given a sample.yml file of:

        ascii_=_symbols: replaced with _
        "ascii_ _controls": dropped (this example uses \t)
        nonascii_א_characters: dropped
        effort_expeñded_tò_preserve_accented_latin_letters: moderate (via unicode NFKD)

then

        yq -o=shell sample.yml

will output

        ascii___symbols='replaced with _'
        ascii__controls='dropped (this example uses \t)'
        nonascii__characters=dropped
        effort_expended_to_preserve_accented_latin_letters='moderate (via unicode NFKD)'

  Encode shell variables: empty values, arrays and maps
Empty values are encoded to empty variables, but empty arrays and maps are skipped.

Given a sample.yml file of:

        empty:
          value:
          array: []
          map:   {}

then

        yq -o=shell sample.yml

will output

        empty_value=

  Encode shell variables: single quotes in values
Single quotes in values are encoded as `“'“’ (close single quote, double-quoted single quote, open single quote).

Given a sample.yml file of:

        name: Miles O'Brien

then

        yq -o=shell sample.yml

will output

        name='Miles O'"'"'Brien'

TOML
Decode from TOML. Note that yq does not yet support outputting in TOML format (and therefore it cannot roundtrip)

  Parse: Simple
Given a sample.toml file of:

        A = "hello"
        B = 12

then

        yq -oy '.' sample.toml

will output

        A: hello
        B: 12

  Parse: Deep paths
Given a sample.toml file of:

        person.name = "hello"
        person.address = "12 cat st"

then

        yq -oy '.' sample.toml

will output

        person:
          name: hello
          address: 12 cat st

  Encode: Scalar
Given a sample.toml file of:

        person.name = "hello"
        person.address = "12 cat st"

then

        yq '.person.name' sample.toml

will output

        hello

  Parse: inline table
Given a sample.toml file of:

        name = { first = "Tom", last = "Preston-Werner" }

then

        yq -oy '.' sample.toml

will output

        name:
          first: Tom
          last: Preston-Werner

  Parse: Array Table
Given a sample.toml file of:

        [owner.contact]
        name = "Tom Preston-Werner"
        age = 36
        [[owner.addresses]]
        street = "first street"
        suburb = "ok"
        [[owner.addresses]]
        street = "second street"
        suburb = "nice"

then

        yq -oy '.' sample.toml

will output

        owner:
          contact:
            name: Tom Preston-Werner
            age: 36
          addresses:
            - street: first street
              suburb: ok
            - street: second street
              suburb: nice

  Parse: Empty Table
Given a sample.toml file of:

        [dependencies]

then

        yq -oy '.' sample.toml

will output

        dependencies: {}

XML
Encode and decode to and from XML. Whitespace is not conserved for round trips - but the order of the fields are.

Consecutive xml nodes with the same name are assumed to be arrays.

XML content data, attributes processing instructions and directives are all created as plain fields.

This can be controlled by:

Flag | Default | Sample XML
--xml-attribute-prefix | + (changing to +@ soon) | Legs in <cat legs="4"/>
--xml-content-name | +content | Meow in <cat>Meow <fur>true</true></cat>
--xml-directive-name | +directive | <!DOCTYPE config system "blah">
--xml-proc-inst-prefix | +p_ | <?xml version="1"?>

{% hint style=“warning” %} Default Attribute Prefix will be changing in v4.30! In order to avoid name conflicts (e.g. having an attribute named “content” will create a field that clashes with the default content name of “+content”) the attribute prefix will be changing to “+@”.

This will affect users that have not set their own prefix and are not roundtripping XML changes.

{% endhint %}

  Encoder / Decoder flag options
In addition to the above flags, there are the following xml encoder/decoder options controlled by flags:

Flag | Default | Description
--xml-strict-mode | false | Strict mode enforces the requirements of the XML specification. When switched off the parser allows input containing common mistakes. See https://pkg.go.dev/encoding/xml#Decoder for more details.
--xml-keep-namespace | true | Keeps the namespace of attributes
--xml-raw-token | true | Does not verify that start and end elements match and does not translate name space prefixes to their corresponding URLs.
--xml-skip-proc-inst | false | Skips over processing instructions, e.g. <?xml version="1"?>
--xml-skip-directives | false | Skips over directives, e.g. <!DOCTYPE config system "blah">

See below for examples

  Parse xml: simple
Notice how all the values are strings, see the next example on how you can fix that.

Given a sample.xml file of:

        <?xml version="1.0" encoding="UTF-8"?>
        <cat>
          <says>meow</says>
          <legs>4</legs>
          <cute>true</cute>
        </cat>

then

        yq -oy '.' sample.xml

will output

        +p_xml: version="1.0" encoding="UTF-8"
        cat:
          says: meow
          legs: "4"
          cute: "true"

  Parse xml: number
All values are assumed to be strings when parsing XML, but you can use the from_yaml operator on all the strings values to autoparse into the correct type.

Given a sample.xml file of:

        <?xml version="1.0" encoding="UTF-8"?>
        <cat>
          <says>meow</says>
          <legs>4</legs>
          <cute>true</cute>
        </cat>

then

        yq -oy ' (.. | select(tag == "!!str")) |= from_yaml' sample.xml

will output

        +p_xml: version="1.0" encoding="UTF-8"
        cat:
          says: meow
          legs: 4
          cute: true

  Parse xml: array
Consecutive nodes with identical xml names are assumed to be arrays.

Given a sample.xml file of:

        <?xml version="1.0" encoding="UTF-8"?>
        <animal>cat</animal>
        <animal>goat</animal>

then

        yq -oy '.' sample.xml

will output

        +p_xml: version="1.0" encoding="UTF-8"
        animal:
          - cat
          - goat

  Parse xml: force as an array
In XML, if your array has a single item, then yq doesn’t know its an array. This is how you can consistently force it to be an array. This handles the 3 scenarios of having nothing in the array, having a single item and having multiple.

Given a sample.xml file of:

        <zoo><animal>cat</animal></zoo>

then

        yq -oy '.zoo.animal |= ([] + .)' sample.xml

will output

        zoo:
          animal:
            - cat

  Parse xml: force all as an array
Given a sample.xml file of:

        <zoo><thing><frog>boing</frog></thing></zoo>

then

        yq -oy '.. |= [] + .' sample.xml

will output

        - zoo:
            - thing:
                - frog:
                    - boing

  Parse xml: attributes
Attributes are converted to fields, with the default attribute prefix `+'. Use ’–xml-attribute-prefix` to set your own.

Given a sample.xml file of:

        <?xml version="1.0" encoding="UTF-8"?>
        <cat legs="4">
          <legs>7</legs>
        </cat>

then

        yq -oy '.' sample.xml

will output

        +p_xml: version="1.0" encoding="UTF-8"
        cat:
          +@legs: "4"
          legs: "7"

  Parse xml: attributes with content
Content is added as a field, using the default content name of +content. Use --xml-content-name to set your own.

Given a sample.xml file of:

        <?xml version="1.0" encoding="UTF-8"?>
        <cat legs="4">meow</cat>

then

        yq -oy '.' sample.xml

will output

        +p_xml: version="1.0" encoding="UTF-8"
        cat:
          +content: meow
          +@legs: "4"

  Parse xml: content split between comments/children
Multiple content texts are collected into a sequence.

Given a sample.xml file of:

        <root>  value  <!-- comment-->anotherValue <a>frog</a> cool!</root>

then

        yq -oy '.' sample.xml

will output

        root:
          +content: # comment
            - value
            - anotherValue
            - cool!
          a: frog

  Parse xml: custom dtd
DTD entities are processed as directives.

Given a sample.xml file of:

        <?xml version="1.0"?>
        <!DOCTYPE root [
        <!ENTITY writer "Blah.">
        <!ENTITY copyright "Blah">
        ]>
        <root>
            <item>&writer;&copyright;</item>
        </root>

then

        yq '.' sample.xml

will output

        <?xml version="1.0"?>
        <!DOCTYPE root [
        <!ENTITY writer "Blah.">
        <!ENTITY copyright "Blah">
        ]>
        <root>
          <item>&amp;writer;&amp;copyright;</item>
        </root>

  Parse xml: skip custom dtd
DTDs are directives, skip over directives to skip DTDs.

Given a sample.xml file of:

        <?xml version="1.0"?>
        <!DOCTYPE root [
        <!ENTITY writer "Blah.">
        <!ENTITY copyright "Blah">
        ]>
        <root>
            <item>&writer;&copyright;</item>
        </root>

then

        yq --xml-skip-directives '.' sample.xml

will output

        <?xml version="1.0"?>
        <root>
          <item>&amp;writer;&amp;copyright;</item>
        </root>

  Parse xml: with comments
A best attempt is made to preserve comments.

Given a sample.xml file of:

        <!-- before cat -->
        <cat>
            <!-- in cat before -->
            <x>3<!-- multi
        line comment
        for x --></x>
            <!-- before y -->
            <y>
                <!-- in y before -->
                <d><!-- in d before -->z<!-- in d after --></d>

                <!-- in y after -->
            </y>
            <!-- in_cat_after -->
        </cat>
        <!-- after cat -->

then

        yq -oy '.' sample.xml

will output

        # before cat
        cat:
          # in cat before
          x: "3" # multi
          # line comment
          # for x
          # before y
          y:
            # in y before
            # in d before
            d: z # in d after
            # in y after
          # in_cat_after
        # after cat

  Parse xml: keep attribute namespace
Defaults to true

Given a sample.xml file of:

        <?xml version="1.0"?>
        <map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url"></map>

then

        yq --xml-keep-namespace=false '.' sample.xml

will output

        <?xml version="1.0"?>
        <map xmlns="some-namespace" xsi="some-instance" schemaLocation="some-url"></map>

instead of

        <?xml version="1.0"?>
        <map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url"></map>

  Parse xml: keep raw attribute namespace
Defaults to true

Given a sample.xml file of:

        <?xml version="1.0"?>
        <map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url"></map>

then

        yq --xml-raw-token=false '.' sample.xml

will output

        <?xml version="1.0"?>
        <map xmlns="some-namespace" xmlns:xsi="some-instance" some-instance:schemaLocation="some-url"></map>

instead of

        <?xml version="1.0"?>
        <map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url"></map>

  Encode xml: simple
Given a sample.yml file of:

        cat: purrs

then

        yq -o=xml sample.yml

will output

        <cat>purrs</cat>

  Encode xml: array
Given a sample.yml file of:

        pets:
          cat:
            - purrs
            - meows

then

        yq -o=xml sample.yml

will output

        <pets>
          <cat>purrs</cat>
          <cat>meows</cat>
        </pets>

  Encode xml: attributes
Fields with the matching xml-attribute-prefix are assumed to be attributes.

Given a sample.yml file of:

        cat:
          +@name: tiger
          meows: true

then

        yq -o=xml sample.yml

will output

        <cat name="tiger">
          <meows>true</meows>
        </cat>

  Encode xml: attributes with content
Fields with the matching xml-content-name is assumed to be content.

Given a sample.yml file of:

        cat:
          +@name: tiger
          +content: cool

then

        yq -o=xml sample.yml

will output

        <cat name="tiger">cool</cat>

  Encode xml: comments
A best attempt is made to copy comments to xml.

Given a sample.yml file of:

        #
        # header comment
        # above_cat
        #
        cat: # inline_cat
          # above_array
          array: # inline_array
            - val1 # inline_val1
            # above_val2
            - val2 # inline_val2
        # below_cat

then

        yq -o=xml sample.yml

will output

        <!--
        header comment
        above_cat
        -->
        <!-- inline_cat -->
        <cat><!-- above_array inline_array -->
          <array>val1<!-- inline_val1 --></array>
          <array><!-- above_val2 -->val2<!-- inline_val2 --></array>
        </cat><!-- below_cat -->

  Encode: doctype and xml declaration
Use the special xml names to add/modify proc instructions and directives.

Given a sample.yml file of:

        +p_xml: version="1.0"
        +directive: 'DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" '
        apple:
          +p_coolioo: version="1.0"
          +directive: 'CATYPE meow purr puss '
          b: things

then

        yq -o=xml sample.yml

will output

        <?xml version="1.0"?>
        <!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
        <apple><?coolioo version="1.0"?><!CATYPE meow purr puss >
          <b>things</b>
        </apple>

  Round trip: with comments
A best effort is made, but comment positions and white space are not preserved perfectly.

Given a sample.xml file of:

        <!-- before cat -->
        <cat>
            <!-- in cat before -->
            <x>3<!-- multi
        line comment
        for x --></x>
            <!-- before y -->
            <y>
                <!-- in y before -->
                <d><!-- in d before -->z<!-- in d after --></d>

                <!-- in y after -->
            </y>
            <!-- in_cat_after -->
        </cat>
        <!-- after cat -->

then

        yq '.' sample.xml

will output

        <!-- before cat -->
        <cat><!-- in cat before -->
          <x>3<!-- multi
        line comment
        for x --></x><!-- before y -->
          <y><!-- in y before
        in d before -->
            <d>z<!-- in d after --></d><!-- in y after -->
          </y><!-- in_cat_after -->
        </cat><!-- after cat -->

  Roundtrip: with doctype and declaration
yq parses XML proc instructions and directives into nodes. Unfortunately the underlying XML parser loses whitespace information.

Given a sample.xml file of:

        <?xml version="1.0"?>
        <!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
        <apple>
          <?coolioo version="1.0"?>
          <!CATYPE meow purr puss >
          <b>things</b>
        </apple>

then

        yq '.' sample.xml

will output

        <?xml version="1.0"?>
        <!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
        <apple><?coolioo version="1.0"?><!CATYPE meow purr puss >
          <b>things</b>
        </apple>

AUTHORS
Mike Farah.