# Share your JupyterLite Examples **URL:** \ **Category:** Development\ **Tags:** workflow\ **Created:** [February 12, 2022, 7:05pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134 "2022-02-12T19:05:19Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Symbioquine](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/symbioquine/32/284_2.png) [@Symbioquine](https://farmos.discourse.group/u/Symbioquine)\ **Post date:** [February 12, 2022, 7:05pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/1 "2022-02-12T19:05:19Z")
I’m creating this topic as a place for folks to share examples and useful snippets related to using [JupyterLite](https://github.com/jupyterlite/jupyterlite) with farmOS. (Possibly via the [JupyterLite Drupal Module](https://www.drupal.org/project/jupyterlite).) Ideally, each example/snippet should include a screenshot and a copy of the notebook. (A GitHub Gist works well for the latter since GitHub will format the notebook nicely.) ---
**Author:** ![Symbioquine](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/symbioquine/32/284_2.png) [@Symbioquine](https://farmos.discourse.group/u/Symbioquine)\ **Post date:** [February 12, 2022, 7:07pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/2 "2022-02-12T19:07:34Z")
## Seeding Logs by Weekday ![image](https://global.discourse-cdn.com/free1/uploads/farm_os/original/1X/c1e5e585649611e67c031c59b7a644835b25be1a.png) > ## Animal CSV Import ![image](https://global.discourse-cdn.com/free1/uploads/farm_os/original/1X/a771a5d878a397a77f3fced5689a72e1abf181c3.png) > ![image](https://global.discourse-cdn.com/free1/uploads/farm_os/original/1X/fd9d42df1bd1646118dbbf98acce837278c7f0e3.jpeg) ---
**Author:** ![mstenta](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/mstenta/32/4_2.png) [@mstenta](https://farmos.discourse.group/u/mstenta)\ **Post date:** [February 12, 2022, 7:21pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/3 "2022-02-12T19:21:51Z")
Great topic idea @Symbioquine! This is exciting stuff! 😀 > [@Symbioquine](#): > > Animal CSV Import Attn: @walt @Farmer-Ed @pat @JustGav @gbathree Cross-reference: [CSV Importers in v2.x](https://farmos.discourse.group/t/csv-importers-in-v2-x/942) ---
**Author:** ![pat](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/pat/32/1050_2.png) [@pat](https://farmos.discourse.group/u/pat)\ **Post date:** [February 12, 2022, 8:49pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/4 "2022-02-12T20:49:56Z")
Nice. Gonna have to dig into this. (This too) ---
**Author:** ![Farmer-Ed](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/farmer-ed/32/662_2.png) [@Farmer-Ed](https://farmos.discourse.group/u/Farmer-Ed)\ **Post date:** [February 12, 2022, 9:00pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/5 "2022-02-12T21:00:23Z")
Cheers @mstenta I’ve installed the module, just have to go and figure out what it can do for me. ---
**Author:** ![mstenta](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/mstenta/32/4_2.png) [@mstenta](https://farmos.discourse.group/u/mstenta)\ **Post date:** [February 12, 2022, 9:25pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/6 "2022-02-12T21:25:39Z")
This YouTube video might help (if you didn’t see it already in the CSV thread, or for newcomers who find this comment)… 🙂 Note: @donblair mentions in the video that farmOS.py doesn’t work, but @paul121 figured out a way to do it… [See this Gist.](https://gist.github.com/paul121/cd64a1e8ef2d609c2f6abd6419420562) [![](https://global.discourse-cdn.com/free1/uploads/farm_os/original/1X/05ae4262f18778557c18f028ed65fe46b0c7a570.jpeg "Jupyterlite and FarmOS -- Proof of Concept") ](https://www.youtube.com/watch?v=87jqCTMwsXk) ---
**Author:** ![Symbioquine](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/symbioquine/32/284_2.png) [@Symbioquine](https://farmos.discourse.group/u/Symbioquine)\ **Post date:** [February 15, 2022, 4:54pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/7 "2022-02-15T16:54:57Z")
## Land KML Import ![image](https://global.discourse-cdn.com/free1/uploads/farm_os/original/1X/797485e05ce14a8cf0702ae2c30d5f15e6cd76fc.png) > The data is the `cb_2018_us_state_20m.kml` file from [https://www2.census.gov/geo/tiger/GENZ2018/kml/cb\_2018\_us\_state\_20m.zip](https://www2.census.gov/geo/tiger/GENZ2018/kml/cb_2018_us_state_20m.zip) ![image](https://global.discourse-cdn.com/free1/uploads/farm_os/original/1X/c18daf372cacb5ec19d07ed2a3a2fad47c079394.jpeg) ---
**Author:** ![walt](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/walt/32/13_2.png) [@walt](https://farmos.discourse.group/u/walt)\ **Post date:** [February 19, 2022, 2:36pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/8 "2022-02-19T14:36:37Z")
Keen as i am to see this JupyterLite API interface coming to an instance near me (i.e. [myinstance.farmos.net/jupyterlite](http://myinstance.farmos.net/jupyterlite) -right @mstenta ? 🥺 ), i have in the meantime been exploring this alternative by Google called “Colaboratory” - first awkward experiments in [this here .ipynb notebook](https://colab.research.google.com/drive/1AtbWfa52LuR6FuXDs2XGVjAhoVBsRppg?usp=sharing). AFAICT, this is essentially a Jupyter Notebook running in the browser (i.e. zero-installation on desktop), but powered by Google’s analytics engine in the cloud, so it’s like the JupyterLite example @donblair shared in that respect. All those DataSci py libes popular in DS/ML circles are there for the `'import'` -e.g. ‘`requests`’ @paul121 -and access to the farmOS.py library is as easy as '`!pip install farmOS==1.0.0b3'`, as you can see in the example linked above. It has some value-add features in the interface like a TOC outliner, and a very cool “Interactive Data Table” extension, which wraps filter/ sort/ pagination controls around the pandas dataframe. And of course (this being Google), they make it easy to share notebooks privately to anyone with a gmail, save it to gDrive, Dropbox, Gitub, gist, etc. All that being said: in the interest of using FOSS whenever possible, i would much rather have my farmOS API interface in Jupyterlite, even without those extra Google features. Just so long as i can run farmOS.py in it- including that `'requests'` library, which is i gather essential -then i will happily to adopt JupyterLite as my API interface, just as soon as it lands in Farmier. ---
**Author:** ![mstenta](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/mstenta/32/4_2.png) [@mstenta](https://farmos.discourse.group/u/mstenta)\ **Post date:** [February 19, 2022, 2:55pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/9 "2022-02-19T14:55:51Z")
@walt You should be able to do whatever you need already! You can use the JupyterLite hosted on EdgeCollective’s (@donblair’s) GitHub Pages: [https://edgecollective.io/jupyterlite/](https://edgecollective.io/jupyterlite/) The only thing that’s necessary to get that to work with Farmier-hosted instances is we need to “bless” (via [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) config) the `edgecollective.io` domain in your instance so that your browser “trusts” the third-party URL. I did that already for yours, so you should be good to go! (If anyone else is using Farmier and wants to play around with [https://edgecollective.io/jupyterlite/](https://edgecollective.io/jupyterlite/) ping me!) Ultimately, it will be great to include the new [Drupal JupyterLite](https://www.drupal.org/project/jupyterlite) module that @Symbioquine and I started on Farmier - but that’s not strictly necessary. The only benefit is you don’t need to configure CORS because it’s serving JupyterLite from the same domain, so your browser will automatically “trust” it. @walt Just be aware of the way “file storage” works with JupyterLite, so you don’t lose anything! _ **All files you upload or create in JupyterLite are stored IN YOUR BROWSER SESSION** _ … they are NOT stored on the farmOS server. If you clear your browser cache or change browsers, they are gone! We have some fun ideas to store/serve files from farmOS itself being discussed here: > **[\[META\] File management](https://www.drupal.org/project/jupyterlite/issues/3264048)** > > Problem/Motivation I'm opening this \[META\] issue to document and discuss some ideas that have come up in chat, with regard to the way JupyterLite "persists" files (or doesn't actually). donblair: i do think that jupyterlite as it currently works may... ---
**Author:** ![walt](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/walt/32/13_2.png) [@walt](https://farmos.discourse.group/u/walt)\ **Post date:** [February 19, 2022, 3:23pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/10 "2022-02-19T15:23:31Z")
> [@mstenta](#): > > @walt You should be able to do whatever you need already! You can use the JupyterLite hosted on EdgeCollective’s (@donblair’s) GitHub Pages: [https://edgecollective.io/jupyterlite/](https://edgecollective.io/jupyterlite/) Thanks, @mstenta -since you configured my instance to accept API calls from that URL- i did have a serious play with this. Works just fine for API access- nice! But when i upload a .CSV to that same directory in which the .ipynb resides, and try to do anything with it, it fails to find the file. Have tried every way i could think to get around this, and consulted @donblair about the problem… No joy ☹ > [@mstenta](#): > > @walt Just be aware of the way “file storage” works with JupyterLite, so you don’t lose anything! _ **All files you upload or create in JupyterLite are stored IN YOUR BROWSER SESSION** _ … they are NOT stored on the farmOS server. If you clear your browser cache or change browsers, they are gone! Good cautionary note, thanks -but not a problem at this point, as my browser cannot lose what it cannot even find in the first place! In fact the browser finds & uploads the file just fine; it’s jLite that can’t seem to find it. > [@mstenta](#): > > We have some fun ideas to store/serve files from farmOS itself being discussed here: Heh: i’ll bet that “some [confused] users” Don mentioned in first line of his problem statement was inspired by yours truly \<8-)… ---
**Author:** ![Symbioquine](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/symbioquine/32/284_2.png) [@Symbioquine](https://farmos.discourse.group/u/Symbioquine)\ **Post date:** [February 19, 2022, 4:22pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/11 "2022-02-19T16:22:48Z")
> [@walt](#): > > Works just fine for API access- nice! But when i upload a .CSV to that same directory in which the .ipynb resides, and try to do anything with it, it fails to find the file. Have tried every way i could think to get around this, and consulted @donblair about the problem… No joy For now, you need this bit of magic that I’ve included in some of my examples above; ```Python # From https://gist.github.com/bollwyvl/132aaff5cdb2c35ee1f75aed83e87eeb async def get_contents(path): """use the IndexedDB API to acess JupyterLite's in-browser (for now) storage for documentation purposes, the full names of the JS API objects are used. see https://developer.mozilla.org/en-US/docs/Web/API/IDBRequest """ import js, asyncio DB_NAME = "JupyterLite Storage" # we only ever expect one result, either an error _or_ success queue = asyncio.Queue(1) IDBOpenDBRequest = js.self.indexedDB.open(DB_NAME) IDBOpenDBRequest.onsuccess = IDBOpenDBRequest.onerror = queue.put_nowait await queue.get() if IDBOpenDBRequest.result is None: return None IDBTransaction = IDBOpenDBRequest.result.transaction("files", "readonly") IDBObjectStore = IDBTransaction.objectStore("files") IDBRequest = IDBObjectStore.get(path, "key") IDBRequest.onsuccess = IDBRequest.onerror = queue.put_nowait await queue.get() return IDBRequest.result.to_py() if IDBRequest.result else None ``` If you put that in the top cell of your notebook (and run it first), then you can access the contents of an “uploaded” (to the browser storage) file named “my\_file.csv” (as a `str` object) with; ```python csv_str = (await get_contents("my_file.csv"))["content"] ``` Or if you _need_ a file object you can wrap that in [io.StringIO](https://docs.python.org/3/library/io.html#text-i-o). e.g. ```python file_obj = io.StringIO((await get_contents("my_file.csv"))["content"]) ``` ---
**Author:** ![mstenta](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/mstenta/32/4_2.png) [@mstenta](https://farmos.discourse.group/u/mstenta)\ **Post date:** [February 19, 2022, 4:34pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/12 "2022-02-19T16:34:43Z")
Oh thanks @Symbioquine! I wasn’t aware of that issue. @walt hopefully that helps! The same issue would have been present on a Farmier-hosted JupyterLite, in either case. 🤷 ---
**Author:** ![Symbioquine](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/symbioquine/32/284_2.png) [@Symbioquine](https://farmos.discourse.group/u/Symbioquine)\ **Post date:** [February 19, 2022, 4:42pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/13 "2022-02-19T16:42:09Z")
Yeah, I think that’s one of the rough edges that will get resolved as JupyterLite moves out of alpha. Maybe as part of [jupyterlite/jupyterlite#315](https://github.com/jupyterlite/jupyterlite/issues/315). ---
**Author:** ![walt](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/walt/32/13_2.png) [@walt](https://farmos.discourse.group/u/walt)\ **Post date:** [February 19, 2022, 4:49pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/14 "2022-02-19T16:49:07Z")
> [@Symbioquine](#): > > `file_obj = io.StringIO((await get_contents("my_file.csv"))["content"])` Thanks @Symbioquine, but… I did put the long magic spell atop the notebook, have tried both of the calls you suggested below it, and neither seems to work. You can see from the below screenshot that my CROPS.csv for upload is in the same directory as the .ipynb… And you can see the error msg when i try the first method. Can you make any sense of this? ![Screenshot 2022-02-19 at 16.46.15](https://global.discourse-cdn.com/free1/uploads/farm_os/original/1X/603bed09543c58994bd92f92124787343c30671b.png) ---
**Author:** ![Symbioquine](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/symbioquine/32/284_2.png) [@Symbioquine](https://farmos.discourse.group/u/Symbioquine)\ **Post date:** [February 19, 2022, 4:54pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/15 "2022-02-19T16:54:08Z")
> [@walt](#): > > And you can see the error msg when i try the first method. Can you make any sense of this? I think I see two problems; - If the cell at the top of the screenshot is the “long magic spell” you copied, I don’t think you got the whole thing. The last line should start with “return IDBRequest” - I think you need to remove the quotes from `df=pd.read_csv("csv_str")`. I think it should be `df=pd.read_csv(csv_str)`. ---
**Author:** ![walt](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/walt/32/13_2.png) [@walt](https://farmos.discourse.group/u/walt)\ **Post date:** [February 19, 2022, 5:09pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/16 "2022-02-19T17:09:51Z")
Thanks @Symbioquine ; fixed the two errors you cite, but i still get the same error- `TypeError: 'NoneType' object is not subscriptable` -on line 2, where the “csv\_str” variable is defined (not on line 4, where it is invoked -now w/o the quotes). Got any more insight into what that error msg might actually refer to? (could be many things, according to google, that would trigger this “most common exception in python”) ---
**Author:** ![walt](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/walt/32/13_2.png) [@walt](https://farmos.discourse.group/u/walt)\ **Post date:** [February 19, 2022, 6:51pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/17 "2022-02-19T18:51:24Z")
PS: problem solved, using this method, following that longer script above in the previous block: ```auto import io file_obj = io.StringIO((await get_contents("myfilename.csv"))["content"]) df=pd.read_csv(file_obj) ``` note to fellow n00bs: you can’t just refer to yourfilename.csv in the script, even tho it’s in the same directory as the .ipynb ; you should grab the path via right-click on the file in Jupyterlite file browser, because it needs an absolute reference (was nested in the farmos/ directory, in this case). Thanks a heap, @Symbioquine , for all the help it took to find my mistake! ---
**Author:** ![walt](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/walt/32/13_2.png) [@walt](https://farmos.discourse.group/u/walt)\ **Post date:** [February 20, 2022, 11:40am UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/18 "2022-02-20T11:40:52Z")
In the course of my stumbling around yesterday, related to this CSV upload challenge, i came to a deeper realisation of what you wrote yesterday @mstenta, i.e.: > Just be aware of the way “file storage” works with JupyterLite, so you don’t lose anything! _ **All files you upload or create in JupyterLite are stored IN YOUR BROWSER SESSION** _ … they are NOT stored on the farmOS server. If you clear your browser cache or change browsers, they are gone! Yes; in fact, to debug my problem, i _had_ to switch from Chrome to Firefox, and of course the subject file had to be re-uploaded… Which put me to wonder about a coupla things, in context of this UseCase: 1. A primary benefit of this application architecture is having the freedom to work from any machine- e.g. farm office and/or home -but, given the diffs that will inevitably arise in state across those two machines, how can we mitigate the confusion that will consequently arise? 2. Memory management: This little (26kb) CSV is small enough to be of no concern… But this being a workflow we plan to run at least weekly- and sometimes on much larger files -what might be the negative impact on browser/system performance, and what should be done to mitigate that problem? 3. Given these (and other?) limitations, a JupyterLite NB that references files is not suitable for sharing a replicable result -whether in the interest of tech support (as @Symbioquine and i experienced yesterday) or in the larger context of Replicable Data Science. Obviously i don’t understand this technology enough… So i did a little digging yesterday, from which i gathered that localStorage is a subtype of [Web Storage](https://en.wikipedia.org/wiki/Web_storage) (f.k.a. DOM Storage), along with two other forms that are more familiar to me. What complicates matters further is the different ways in which browser-makers implement the standard (that’s the point at which my head started to hurt, so i quit digging), but i did find this little table (pictured below) that helped me to understand essential similarities & diffs. Bottom-line: There’s enough deep voodoo about this stuff that- to avoid sliding into even deeper doodoo! -i think it will be wise to store any files referenced in the JupyterLite NB in an online archive, and link them explicitly in the document. ![Screenshot 2022-02-20 at 11.01.30](https://global.discourse-cdn.com/free1/uploads/farm_os/original/1X/061cb40581c21e016a5daf0b8fea5424beea4c07.png) ---
**Author:** ![Symbioquine](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/symbioquine/32/284_2.png) [@Symbioquine](https://farmos.discourse.group/u/Symbioquine)\ **Post date:** [February 20, 2022, 3:26pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/19 "2022-02-20T15:26:32Z")
Hey @walt [this article](https://blog.bitsrc.io/browser-storage-options-and-their-state-in-2021-6808b949087a) is a little better because it’s a little more up to date and includes IndexedDB, which is the storage API which JupyterLite will use in most browsers. > ![image](https://global.discourse-cdn.com/free1/uploads/farm_os/original/1X/f159c2777638713d7d3ca9b10846a9a2713522d3.png) > > _A more up-to-date table from that article._ Otherwise, I think you’re making some great points - most of which don’t have concrete answers. I will say though that many of those issues are mitigated just by changing how we think of the “storage” in JupyterLite. If we consider the storage in JupyterLite like a sandbox environment or temporary work area, then we can treat those things as advantages. > [@walt](#): > > Given these (and other?) limitations, a JupyterLite NB that references files is not suitable for sharing a replicable result -whether in the interest of tech support (as @Symbioquine and i experienced yesterday) or in the larger context of Replicable Data Science. I would argue that it is better for both the tech support and replicable data science scenarios. - I was able to start with a clean slate and bring in just the files I needed to try and reproduce the problem you were having. - I was able to modify the files without any risk losing data or breaking things for anyone else. - It forces me to be intentional about sharing just the versions/changes which are important. - Conversely, it helps ensure the collective workspace isn’t increasingly littered with semi-relevant experiments. I would also argue that one of the foundations of truly replicable data science is going to be consistent and disciplined use of version control technologies like Git to manage the source code - and in some cases sample data. Perhaps in the future JupyterLite could help with that part, but in a way it’s kind of beautiful that it doesn’t. It’s job is just to be a place that’s reproducible (but not replicated) between users to run some scripts in a little more interactive environment than a text editor and a command line. ---
**Author:** ![walt](https://yyz2.discourse-cdn.com/free1/user_avatar/farmos.discourse.group/walt/32/13_2.png) [@walt](https://farmos.discourse.group/u/walt)\ **Post date:** [February 20, 2022, 3:59pm UTC](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134/20 "2022-02-20T15:59:24Z")
Thanks @Symbioquine for providing a more nuanced perspective. I can see how what i was inclined to view as bugs might be considered features… _Just so long as we_ (a) treat it as a “sandbox,” and (b) employ that “more consistent and disciplined use of version control tech” for both sources and sample data. Also: since you had me open Developer Tools yesterday (in search of that CROPS.csv file), and as that article you linked explains in more detail (illustrated by screenshot below), i can now easily navigate to where these files are stored (IndexedDB indeed, in both Chrome and Firefox), confirm the keys and drill down into values… But only by navigating the JSON tree, in which form my nice tidy tables has been rendered. One day i hope to get over my allergy to these so-deeply-nested JSON trees; still, being more of a rows&columns kinda guy, i have to ask: is there any easy way to translate this JSON tree back into tabular .CSV form? ![](https://global.discourse-cdn.com/free1/uploads/farm_os/original/1X/9cf8bebba1234024781a962bbb4aa8562f659cf4.png) [Next page](https://farmos.discourse.group/t/share-your-jupyterlite-examples/1134.md?page=2)