Exploring v2 API via browser (Firefox)

Development
1

Trying to bend this neat little Python script from @paul121 to my own practical purposes, my efforts have been largely frustrated on account of my lack of understanding about the farmOS v2 data model.

Have browsed the online docs: nice, but insufficiently detailed. Have also browsed (using Firefox; can’t do this in Safari or Chrome apparently) the API of my farm’s instance: another nice hi-level view, but when i try to click thru to details from any of those endpoints, i get told "The current user is not allowed to GET the selected resource." . So: following the 4-step process outlined on Authentication page, i went to the /oauth/authorize endpoint of my instance, filled out the ID & Pwd fields of popup modal window with codes I KNOW to be good, but all i ever get in return is another request for codes, no matter how many times I try this (multiple tries with 2 different admin-level account codes, and always the same result).

SO: Presuming this to be the best method available to access a comprehensive (& comprehensible!) view of farmOS v2 data model[1], can somebody please give me the open sesame for such access?
NB: I don’t even need a write-privileged access to API of my own instance; read-only access to any old instance would do me just fine, since they all share a common database schema, presumably.

[1] If there be some other form of data model documentation available, I will take whatever I can get at this point! (Must be a quirk of my advanced age, but I do pine for the good old days when there was a ERD to be had for every relational database :face_with_monocle: )

1 Like
2

@walt This probably just means you aren’t logged into the site. If I try to browse the /api/asset/animal endpoint as an anonymous user (not logged in), I see the same thing. But if I log in first and then refresh the API endpoint in my browser I can see everything.

3

Dooh! (/me wipes egg off face)
Never occurred to me that this API interface would have anything to do w/ farmOS’s web UI. Thanks, @mstenta!

2 Likes
4

Great! Yea that should make things easier for ya! :smiley:

On a related note… I just learned recently that the schema endpoints are 100% public, which is kinda neat.

For example, you can go to /api/asset/animal/resource/schema (without being logged in) to see the JSON Schema definition of the endpoints.

This may also help in your understanding of the API. JSON Schema basically provides meta information about the API endpoints themselves, so you can understand what each piece is, what type of data it expects, etc. It can be used programmatically to validate JSON documents before they are sent to the server too.

1 Like
5

Even better! This enables free & open collaboration about developments, without all the hassle involved in Authentication.

Is this a feature common to all projects that implement the OpenAPI spec, i wonder?

To what extent (if any) does farmOS implement the full OpenAPI spec?

1 Like
6

"There’s a module for that!"™

We don’t currently include this with farmOS, but it’s easy enough to drop in and turn on.

There are companion modules that will add a UI on top as well:

… however IIRC there was some complexity in setting them up, and they were a bit weird/buggy, so I decided it didn’t make sense to include them in farmOS (at least not yet), but that downstream users could experiment with them.

Do you have experience working with OpenAPI? I don’t have much myself, apart from what @paul121 added to the farmOS Aggregator, and the early tests I did with the above modules…

I wonder if it’s possible to load the OpenAPI information into a separate UI/app? Then perhaps we could just install the openapi module, but not the corresponding UI modules (which were the complicated/buggy parts I think). I’m not really familiar with what’s possible.

7

Sorry; i have no experience of OpenAI myself.

I do however remember all-too-well those years of frustration i wasted, trying to work with data (my data!) locked away in proprietary RDBMS (e.g. Oracle)… And then the freedom of migrating to PostgreSQL-based FOSS with freely shared data models, which gave me the map i needed to get what i wanted out of our database with my choice of nice user-friendly SQL query tools.

To be frank: i still don’t understand why i can’t be granted a read-only view of the Postgres backend of my farmOS v2a instance. Ignorant as i am of the security ramifications, i just know that my database administrator back in them good ol’ days had no fear of me causing any trouble, given the limited db view i was granted. Reflecting on that experience, i did sometimes send a query that either took forever to return results, or timed out, or returned an error msg… But in no case did that hinder in any way the OLTP ops that were going on back in the DP boiler room (where we had a gang of cubicle slaves keying-in data from handwritten forms pretty much around the clock. Good ol’ days indeed, eh? :grin:

2 Likes
8

That’s just a limitation of Farmier hosting. I take security very seriously, so everything is as “locked down” as possible as a first step. Then, if there’s demand for things, I can open them up. But it’s a process to think it all through… and as you say, even read-only access can cause problems.

2 Likes
9

Exploring the API via Firefox browser has been interesting… But navigating these loooong JSON trees is for me still a rather frustrating experience.

So i was pretty happy to have discovered Postman.com this week -all the more so, since @paul121 shared his farm_postman collection. For an experienced developer, the Postman interface may feel a bit clunky, but for me, it seems a nicely structured environment for saving (and documenting!) API calls for easy reuse, with the ability to set certain arguments (e.g. API keys!) as variables, testing out new scripts, etc… But it’s still JSON that gets delivered in the end, alas.

There is however a nice alternative way to view JSON results, implemented via this visualize tab in the Response section of Postman UI, which enables output in the form of tables, charts, graphs, heatmaps, etc… Anything that can be presented via HTML, essentially. It’s all explained and demonstrated quite nicely on this page, and while i have been able to generate such displays from other APIs, i just can’t seem to get even the simplest request & display template to work against the v2 farmOS API.

Following those instructions, i have posted to “Tests” tab of scripting interface the following skeleton script with the simplest possible HTML table, to display just 2 variables (‘name’ and ‘created’) that are common to all data entities that i am interested to query… But, beyond the header row of this table, i can’t get any data to show up, tho i do get the results of this GET request (from {{host}}/api/user/user for example) in JSON form.

Can anyone see the flaw in this logic, i wonder? If you can point me to it, i’d be much obliged!

/walt

let response = pm.response.json()
console.log(response)   
const template = `
    <table>
        <tr>
            <th>Name</th>
            <th>Created</th>
        </tr>
        {{#each response.data.attributes}}
            <tr>
                <td>{{name}}</td>
                <td>{{created}}</td>
            </tr>
        {{/each}}
    </table>
`;

pm.visualizer.set(template, response);
2 Likes
10

PS to my last: Thanks to StackOverflow user PDHide, this one is solved, as follows below.

NB: no attempt yet to “prettify” that HTML table, but this should serve as a minimal script that works against most any entity in farmOS v2, AFAICT. /walt

let farmdata = pm.response.json()
console.log(farmdata)   
let template =
    `
 <table bgcolor="#FFFFGF">
        <tr>
            <th>Name</th>
            <th>Created</th>
        </tr>

        {{#each data}}
            <tr>
                <td>{{attributes.name}}</td>
                <td>{{attributes.created}}</td>
            </tr>
        {{/each}}
    </table>
`
pm.visualizer.set(template, farmdata);
1 Like
11

Way cool! I didn’t know about this but looks useful. Just be careful to not take it too far or else you’ll end up re-creating the tables farmOS already has… ha! :wink:

Feel free to contribute back to that repo I started if it makes sense - exporting these postman collections is kinda confusing so it might be easier to just start your own. I obviously haven’t updated that in over a year.

1 Like
12

Heh -no chance of me ever taking it that far! I’m just using this tool to find my way around the API, using GET requests only at this point, so when i start having to POST or PUT data (which point is coming soon!), i won’t be making a mess of the database!

It’ll be some time before i have anything worthy of sharing for download. I did at least manage to “prettify” the table a bit, but- as you can see in the attached image- that Parent column i added is not giving the desired result, because i still don’t understand how to parse out elements from different branches of the JSON tree. In this example of Land assets, although terms ‘name’ and ‘created’ are attributes in the data table, the ‘parent’ variable is nested under ‘relationships’ branch… But swapping ‘relationships’ in place of its sister ‘attributes’ branch doesn’t work, obviously. Any idea what would be the right way to do this?

postman_1
postman_11682×747 183 KB

let farmdata = pm.response.json()
console.log(farmdata)   
var template = `
    <style type="text/css">
        .tftable {font-size:14px;color:#333333;width:100%;border-width: 1px;border-color: #87ceeb;border-collapse: collapse;}
        .tftable th {font-size:18px;background-color:#87ceeb;border-width: 1px;padding: 8px;border-style: solid;border-color: #87ceeb;text-align:left;}
        .tftable tr {background-color:#ffffff;}
        .tftable td {font-size:14px;border-width: 1px;padding: 8px;border-style: solid;border-color: #87ceeb;}
        .tftable tr:hover {background-color:#e0ffff;}
    </style>
    
    <table class="tftable" border="1">
        <tr>
            <th>Name</th>
            <th>Created</th>
            <th>Parent</th>
        </tr>
        
        {{#each data}}
            <tr id=row_{{@key}} onClick="handleClick(this.id)">
                <td id={{@key}}>{{attributes.name}}</td>
                <td>{{attributes.created}}</td>
                <td>{{relationships.parent}}</td>
            </tr>
        {{/each}}
    </table>
`;

pm.visualizer.set(template, farmdata);
13

For reference, this is a distinction that the JSON:API standard specification makes between “attributes” (simple/direct data fields) and “relationships” (references to other records).

https://jsonapi.org/format/#document-resource-object-attributes

https://jsonapi.org/format/#document-resource-object-relationships

The official specification docs (although a bit dense) are super helpful for understanding how the JSON is structured. Drupal made the decision to adopt JSON:API as the standard for its built-in REST APIs after a long community discussion.

14

Thanks @mstenta for the pointer to relevant docs. Dense indeed! I did nonetheless try to find the key in there to unlock what i’m looking for -in this case of Land Assets, it is the Parent variable- and the section that seemed most pertinent is this:

5.2.7. Resource Links: A server MUST respond to a GET request to the specified URL with a response that includes the resource as the primary data.

Great: so i should be able to access the actual resource -the Land asset in which the subject (i.e. child) Land asset is embedded- but this doesn’t tell me how to call it out in code. In the example above, where it calls for {{relationships.parent}}, which pulls just the same [object Object] text in every case, i found that by elaborating the path to {{relationships.parent.links.related.href}}, i can get a unique URL for each row in the table… But that’s just another inscrutable JSON object, while what i really want is a human readable ‘name’ , or browsable ‘link’ to the asset in farmOS UI, or ideally both (i.e. a name that is a live http ref to the page of that asset).

Can you tell me how to modify that line of code to get what i want? or is this asking for something the system cannot provide?

1 Like
15

@walt I must admit I haven’t dug very deep myself, and haven’t played with the advanced functionality that JSON:API provides very much. There are some decent docs on drupal.org that might help though…

Here is one that might be relevant to your question: Includes | JSON:API module | Drupal Wiki guide on Drupal.org

Here’s the root of the JSON:API drupal.org docs: JSON:API module | Core modules | Drupal Wiki guide on Drupal.org

1 Like
16

I think @mstenta is right, you could use the JSON:API includes feature to pull in the parent name attribute. But then trying to pull that into your table will be a bit complicated… if you can include JS it might be easier, but not sure if it’s possible with just that templating syntax.

If you can get the drupal_internal__id attribute then you could hard code a link to the asset or log, something like: <a href="url/log/{{ attributes.drupal_internal__id }}">{{ attributes.name }}</a>

2 Likes
17

Thanks @paul121 @mstenta for the pointer about that “Includes” module in JSON:API; using it, i’ve been able to pull either ID or drupal_internal_id for related records and pull them into the table for that entity to which these keys serve as Foreign Key -i guess!

But, just to be clear: it is the ‘drupal_internal_id’ attribute -NOT the ‘id’ that uniquely identifies the whole set of attributes in which ‘drupal_internal_id’ is contained- which must be used in order to access any of those other attributes, such as ‘name’?

If so, i’m surprised, because drupal_internal_id (a short integer) is not the one named in relationships; instead it is the higher-level ‘id’ (a long alphanumeric string peppered with hyphens) that occupies this place where one would expect the FK to be. Could be my RDBMS mind-set is more a hindrance than a help here, but i am really struggling to grok this data model!

1 Like
18

@walt By default the API uses the “UUID” (Universally Unique Identifier) of all records as the primary id.

The drupal_internal__id is the id column in the database, which is an auto-incrementing integer.

The main difference is that the UUID will be unique across multiple farmOS instances, whereas the internal ID will only be unique to the current database. So you can think of the UUID as the “global” ID of this record, and the internal ID as the “local” ID, in a way.

But, just to be clear: it is the ‘drupal_internal_id’ attribute -NOT the ‘id’ that uniquely identifies the whole set of attributes in which ‘drupal_internal_id’ is contained- which must be used in order to access any of those other attributes, such as ‘name’?

No, when you are using the farmOS API, it is the UUID that will be used in all references between records/resources.

When you are writing PHP module code, which is at a lower level, then you are typically working with internal IDs.

1 Like
19

This makes perfect sense; my faith in wisdom of core Drupal architecture is restored :grin:

1 Like