I’ve had more than a few times where I’ve thought to myself, “I really just want to take this source dict and make a new dict with its elements without having to have the source dict available yet.”

The Premise

Basically, the idea was to, if you have a dict like:

1
2
3
4
5
6
7
8
9
10
11
12
{
    "author": {
        "name": "FxChiP",
        "id": 15,
        "email": "test@example.org"
    },
    "post": {
        "title": "Hello World",
        "link": "https://fxchip.net/hello-world/",
        "content": "..."
    }
}

And you wanted to just map to something like:

1
2
3
4
5
{
    "post_title": "Hello World",
    "post_content": "...",
    "author_name" :"FxChiP"
}

That you should be able to just define the mapping up front for readability:

1
2
3
4
5
6
7
8
import shapyro
import requests

mapper = shapyro.Template({
    "post_title": shapyro.Get['post']['title'],
    "post_content": shapyro.Get['post']['content'],
    "author_name": shapyro.Get['author']['name']
})

And then just apply it later:

1
result_map = mapper(requests.get("https://fxchip.net/hw.json").json)

The Implementation

At its core, it’s largely just a builder of a chain of lambdas. I was largely inspired by Riemann’s concept of arbitrary functions as streams – Riemann configs are built of essentially some functions that take some parameters and yield a function that applies those parameters to an incoming event. Shapyro operates largely the same way, except that instead of an “event” it’s really just “any particular kind of arbitrary data”. (Which, technically, events kind of are, but that’s a subject for another time)

Probably the real advantage is the way shapyro.Get works though: since you just define the accesses you want to make to the source object against shapyro.Get itself, the underlying class, _GetChainLink, will just proxy those accesses into a function that is performed on either the input object or the result of the last access to the input object. Since all of the accesses are just done by a callable that gets defined by the class, we don’t even have to rely on eval, so we don’t have to do a ton of input validation – and the access itself is done the same way it was requested, so whatever exceptions it might have thrown will be thrown exactly the same way, just when the map is performed rather than immediately.

Recently I added in some stuff where it will check the result of the last thing and, if it’s an asyncio coroutine, will return a coroutine of its own so that the ultimate resolution can be done by just awaiting the return.

Where to get it

I’ve actually uploaded it to pypi! You should be able to just pip install it and import shapyro. Documentation is at the repo; I might have to write some pages more formally, but everything should at least have a __help__. There is also a GitHub repository.