How to access deeply nested JSON data using Go (lang)?

Most often developer needs to consume JSON data from other service and query over them. Querying JSON document is little time-consuming. For the last few days, I was working on a package for Golang to query JSON data easily. The idea and inspiration come from PHP-JSONQ by Nahid Bin Azhar.

Let's take a sample JSON data to start with:

{
   "name":"computers",
   "description":"List of computer products",
   "vendor":{
      "name":"Star Trek",
      "email":"info@example.com",
      "website":"www.example.com",
      "items":[
         {"id":1, "name":"MacBook Pro 13 inch retina","price":1350},
         {"id":2, "name":"MacBook Pro 15 inch retina", "price":1700},
         {"id":3, "name":"Sony VAIO", "price":1200},
         {"id":4, "name":"Fujitsu", "price":850},
         {"id":5, "name":"HP core i5", "price":850, "key": 2300},
         {"id":6, "name":"HP core i7", "price":950},
         {"id":null, "name":"HP core i3 SSD", "price":850}
      ],
      "prices":[
         2400,
         2100,
         1200,
         400.87,
         89.90,
         150.10
     ]
   }
}

Let's find a deeply nested property and handle error properly, in this case, we'll try to access name from the second element of items array, note: items is a property of vendor object. See the example below:

package main

import (
    "fmt"
    "log"

    "github.com/thedevsaddam/gojsonq"
)

func main() {
    jq := gojsonq.New().File("./sample-data.json")
    res := jq.Find("vendor.items.[1].name")

    if jq.Error() != nil {
        log.Fatal(jq.Errors())
    }

    fmt.Println(res)
}

Yahooooo! Very simple right? It looks like working with ORM of JSON data. Let's see some more example to query over the sample data.

Example 1

Query: select * from vendor.items where price > 1200 or id null

Using gojsonq we can do the query like:

package main

import (
    "fmt"

    "github.com/thedevsaddam/gojsonq"
)

func main() {
    jq := gojsonq.New().File("./sample-data.json")
    res := jq.From("vendor.items").Where("price", ">", 1200).OrWhere("id", "=", nil).Get()
    fmt.Println(res)
    // output: [map[price:1350 id:1 name:MacBook Pro 13 inch retina] map[id:2 name:MacBook Pro 15 inch retina price:1700] map[id:<nil> name:HP core i3 SSD price:850]]
}

Example 2

Query: select name, price from vendor.items where price > 1200 or id null

Using gojsonq we can do the query like:

package main

import (
    "fmt"

    "github.com/thedevsaddam/gojsonq"
)

func main() {
    jq := gojsonq.New().File("./sample-data.json")
    res := jq.From("vendor.items").Where("price", ">", 1200).OrWhere("id", "=", nil).Only("name", "price")
    fmt.Println(res)
    // output: [map[name:MacBook Pro 13 inch retina price:1350] map[name:MacBook Pro 15 inch retina price:1700] map[name:HP core i3 SSD price:850]]
}

Example 3

Query: select sum(price) from vendor.items where price > 1200 or id null

Using gojsonq we can do the query like:

package main

import (
    "fmt"

    "github.com/thedevsaddam/gojsonq"
)

func main() {
    jq := gojsonq.New().File("./sample-data.json")
    res := jq.From("vendor.items").Where("price", ">", 1200).OrWhere("id", "=", nil).Sum("price")
    fmt.Println(res)
    // output: 3900
}

Example 4

Query: select price from vendor.items where price > 1200

Using gojsonq we can do the query like:

package main

import (
    "fmt"

    "github.com/thedevsaddam/gojsonq"
)

func main() {
    jq := gojsonq.New().File("./sample-data.json")
    res := jq.From("vendor.items").Where("price", ">", 1200).Pluck("price")
    fmt.Println(res)
    // output: [1350 1700]
}

Example 5

Query: select * from vendor.items order by price

Using gojsonq we can do the query like:

package main

import (
    "fmt"

    "github.com/thedevsaddam/gojsonq"
)

func main() {
    jq := gojsonq.New().File("./sample-data.json")
    res := jq.From("vendor.items").SortBy("price").Get()
    fmt.Println(res)
    // output: [map[id:<nil> name:HP core i3 SSD price:850] map[id:4 name:Fujitsu price:850] map[id:5 name:HP core i5 price:850 key:2300] map[id:6 name:HP core i7 price:950] map[id:3 name:Sony VAIO price:1200] map[id:1 name:MacBook Pro 13 inch retina price:1350] map[id:2 name:MacBook Pro 15 inch retina price:1700]]
}

Example 6

Using gojsonq You can handle errors properly, see the code snippet below:


package main

import (
    "log"

    "github.com/thedevsaddam/gojsonq"
)

func main() {
    jq := gojsonq.New().File("./invalid-file.xjsn")
    err := jq.Error()
    if err != nil {
        log.Fatal(err)
        // 2018/06/25 00:48:58 gojsonq: open ./invalid-file.xjsn: no such file or directory
        // exit status 1
    }
}

Example 7

Let's assume we have a JSON document like this one

{
  "users":[
    {
      "id":1,
      "name":{
        "first":"John",
        "last":"Ramboo"
      }
    },
    {
      "id":2,
      "name":{
        "first":"Ethan",
        "last":"Hunt"
      }
    },
    {
      "id":3,
      "name":{
        "first":"John",
        "last":"Doe"
      }
    }
  ]
}

We want to run a query like this:

Query: select * from users where name.first=John

Using the package you can do the query easily, see the code snippet below:

package main

import (
    "fmt"

    "github.com/thedevsaddam/gojsonq"
)

func main() {
    jq := gojsonq.New().File("./data.json")
    res := jq.From("users").WhereEqual("name.first", "John").Get()
    fmt.Println(res) //output: [map[id:1 name:map[first:John last:Ramboo]] map[id:3 name:map[first:John last:Doe]]]
}

You can access nested level property using DOT (.) for methods like Where/GroupBy/SortBy etc

Note: There are some other useful methods to make life easier! If you like the package do not forget to share with your community and star the repository

Thank you very much for reading this article and don't forget to give your feedback on comment :)

Write your comment…

Word of caution, this is nothing close of an ORM. An ORM/ODM is all about modeling your data, and providing abstractions to do CRUD operations on it in an elegant way. If you were to compare this method to something else, it would be lodash's at() method, Immutable.js get() or pretty much any functional method allowing you to traverse an object from a string representation of a path.

Yes, ORM/ODM is all about modeling data and providing abstractions to do work with the data, I am not sure it's related to CRUD. It could be read/access the data, something look like DAL. What I actually mean is the usages of ODM could be a subset of CRUD

Thanks for your catch, I'll try to update it :)

Reply to this…