NW.js cross platform getting started app

Google Summer of Code
1

Hello,

My name is Midhul Varma, and I’m a Computer Science undergraduate from India. I’m really interested in working on the NW.js based cross-platform getting started app. I’m excited about the large impact it can have on new BB users all over the world. I have some experience working in javascript and also a little in working with node.js.
I forked the incomplete version of the getting started app, and am reading and understanding the code, while simultaneously learning about sockets and BoneScript.

I’ve done some research about flashing via USB and programming SD cards. For flashing through USB in linux, I think the BBBlfs flash script can be directly executed using a system call from node. (I’m wondering about the challenges one might face while doing this). Also, I would like some guidance/suggestions on what can be done for windows and OSX systems.
For programming SD cards, I wasn’t able to find any node modules which could do the task. So I guess it has to be done by making system calls. On linux this can be done easily using the dd command and on windows we could integrate the Flashnul package (which is licensed under GPL: http://shounen.ru/soft/flashnul/readme.rus.html) for the same.

I want some guidance as to how I could implement these features and what challenges I could face on the way. I’d also like to know about other feature ideas which can be implemented in this project. (apart from getting started content, browsing, driver installation, latest image downloading, flashing via usb and sd card programming)

2

Hello,

My name is Midhul Varma, and I’m a Computer Science undergraduate from India. I’m really interested in working on the NW.js based cross-platform getting started app. I’m excited about the large impact it can have on new BB users all over the world. I have some experience working in javascript and also a little in working with node.js.
I forked the incomplete version of the getting started app, and am reading and understanding the code, while simultaneously learning about sockets and BoneScript.

I’ve done some research about flashing via USB and programming SD cards. For flashing through USB in linux, I think the BBBlfs flash script can be directly executed using a system call from node.

Probably invoking via ‘system’ call is fine, but the stdio channel should have some clear specification for on-going status updates, error reporting, etc. Other approaches could be valid as well, as long as you can show how you can make a valid project plan and have ideas how you can tackle the architectural challengs.

(I’m wondering about the challenges one might face while doing this).

Having clearly defined way for dealing with errors, tracking updates and controlling with some flexibility are probably the big sticking points.

Also, I would like some guidance/suggestions on what can be done for windows and OSX systems.

Do you have any of these systems to play with to try addressing the concerns? Vlad might have some thoughts and I think some are noted in the BBBlfs repository issues.

For programming SD cards, I wasn’t able to find any node modules which could do the task. So I guess it has to be done by making system calls. On linux this can be done easily using the dd command and on windows we could integrate the Flashnul package (which is licensed under GPL: http://shounen.ru/soft/flashnul/readme.rus.html) for the same.

Cool.

I want some guidance as to how I could implement these features and what challenges I could face on the way. I’d also like to know about other feature ideas which can be implemented in this project. (apart from getting started content, browsing, driver installation, latest image downloading, flashing via usb and sd card programming)

Have you seen BBUI and bonecard demos at all?

3

Thank you for your attention.
I looked into the BBUI and bonecard demos as you suggested.

My vision for how this project would look is something like this:
Bone101 is essentially built in the form of a website with hyperlink navigation and relatively long pages. Here we have a desktop application, so the getting started content can be put into a re-modeled UI which will be more suitable for a desktop application and will give users more clear and step-by-step information. It could be something like allowing different modules(like cloud9 ide, sd programmer etc…) to open in new windows. (I guess nw.js gives us good flexibility for UI customization)
So, BBUI and a bonecard deck could appear as separate modules on the desktop app.
Regarding bonecards, after looking at the prototype in the Bone101 repository this is what I understood: They are simply a nice way by which users can create demos for BB, by simply putting up the code on a gist. Please correct me if I have missed something.

I also had this other idea for the project in mind:
Normally the Cloud9 IDE is served from the BB, so users have to be have their BBs connected to code on it. If we could integrate the IDE separately into the NW.js app, it would enable users to code without the BB connected and later quickly upload the code after connecting. (somewhat like having an offline IDE). I did some searching about the integration. It looks like integrating the entire IDE may be difficult, as there seem to be a large number of dependencies (like python and g++) and that would make the installation process more difficult on the users end (which is clearly a big no no). But maybe we could integrate only the essential features, or maybe separately implement file system access and some other small features on top of something like the Ace editor (http://ace.c9.io/), to have a light-weight custom IDE.
What is your take on this?

4

Further updates:

I did some more research about writing images to micro SD cards through the nw.js app. It turns out that it is almost impossible to write a single cross-platform procedure which can do the task on all OS. The best option seems to be this:
First the downloaded image file is sent to the BB over USB, then the app talks to the board over SSH giving it commands to program the microSD card inserted into the board, while keeping track of progress through the stdout and stderr streams. Vlad told me that he was doing something similar to this in an older version of BBBlfs, though he was talking to the board via serial communication instead. We could send commands over shh from node using https://github.com/mscdex/ssh2 or something similar. This will keep it uniform across all platforms. (unlike serial communication which seems to be more platform and driver dependant)

I also programmed a prototype nw.js app which can write images to SD cards connected to linux systems. Link: https://github.com/webglider/protosd. The app takes care of the entire writing process and the user wont have to touch the terminal. It demonstrates how the process can be controlled from node

Please let me know your thoughts regarding this asap so that I can start working on my proposal.

Thanks

5

Further updates:

I did some more research about writing images to micro SD cards through the nw.js app. It turns out that it is almost impossible to write a single cross-platform procedure which can do the task on all OS. The best option seems to be this:

I think the task is really to make the equivalent of the BBBlfs boot stage to work to booting a Linux kernel over USB for each of Linux, Windows and Mac OS X hosts. Putting this under nw.js should be simple after that application is known to exist.

First the downloaded image file is sent to the BB over USB, then the app talks to the board over SSH giving it commands to program the microSD card inserted into the board,

The first image is likely to be either u-boot or Linux. Both provide mass-storage class support over USB which is supported on all major operating systems.

Is ‘dd’ over USB so hard to make cross-platform?

Providing a custom over-network protocol for ‘dd’ might be most practical, but you should try to look to see what the state of the art is.

Googling finds:

while keeping track of progress through the stdout and stderr streams. Vlad told me that he was doing something similar to this in an older version of BBBlfs, though he was talking to the board via serial communication instead. We could send commands over shh from node using https://github.com/mscdex/ssh2 or something similar. This will keep it uniform across all platforms. (unlike serial communication which seems to be more platform and driver dependant)

ssh may or may not be necessary. A simple TCP socket might do fine.

I also programmed a prototype nw.js app which can write images to SD cards connected to linux systems. Link: https://github.com/webglider/protosd. The app takes care of the entire writing process and the user wont have to touch the terminal. It demonstrates how the process can be controlled from node

I did a slightly similar UI at BeagleBoard.org - updater

I have 2 fundamental challenges with the above implementation:

  • The Bone itself is required to have network access because it was the one pulling down the image. The browser doesn’t provide an easy mechanism for fetching data from one host and pushing it to another due to cross-site scripting risks, at least by my understanding. There’s probably some kind of work-around, but writing a nw.js app would get around this limitation.
  • The Bone can’t write to the eMMC if booted from the eMMC. Instead, we’d need to get an image booting that is running out of RAM. I suggest booting a ‘buildroot’ image. This, in my mind, is the number 1 challenge to the whole project… we simply need a way to boot over USB reliably from every operating system. After that, everything is downhill.
6

Hello all,

Further updates:

I did some more research about writing images to micro SD cards through the nw.js app. It turns out that it is almost impossible to write a single cross-platform procedure which can do the task on all OS. The best option seems to be this:

I think the task is really to make the equivalent of the BBBlfs boot stage to work to booting a Linux kernel over USB for each of Linux, Windows and Mac OS X hosts. Putting this under nw.js should be simple after that application is known to exist.

I described the challanges of both Windows/OS X for BBBlfs. For windows we need some inside info from TI, if they tested the usb booting on Windows (WinUSB driver does not work with the AM335x chip). For OS X we need a codeless kext (check: https://github.com/ungureanuvladvictor/BBBlfs/issues/16). If we do this, as Jason said the rest is easy just wrapping it into a nw.js app.

First the downloaded image file is sent to the BB over USB, then the app talks to the board over SSH giving it commands to program the microSD card inserted into the board,

The first image is likely to be either u-boot or Linux. Both provide mass-storage class support over USB which is supported on all major operating systems.

Is ‘dd’ over USB so hard to make cross-platform?

Providing a custom over-network protocol for ‘dd’ might be most practical, but you should try to look to see what the state of the art is.

Googling finds:

I suggest here to take the g_mass_storage approach and just use dd from the host.

while keeping track of progress through the stdout and stderr streams. Vlad told me that he was doing something similar to this in an older version of BBBlfs, though he was talking to the board via serial communication instead. We could send commands over shh from node using https://github.com/mscdex/ssh2 or something similar. This will keep it uniform across all platforms. (unlike serial communication which seems to be more platform and driver dependant)

ssh may or may not be necessary. A simple TCP socket might do fine.

I also programmed a prototype nw.js app which can write images to SD cards connected to linux systems. Link: https://github.com/webglider/protosd. The app takes care of the entire writing process and the user wont have to touch the terminal. It demonstrates how the process can be controlled from node

I did a slightly similar UI at http://beagleboard.org/updater

I have 2 fundamental challenges with the above implementation:

  • The Bone itself is required to have network access because it was the one pulling down the image. The browser doesn’t provide an easy mechanism for fetching data from one host and pushing it to another due to cross-site scripting risks, at least by my understanding. There’s probably some kind of work-around, but writing a nw.js app would get around this limitation.
  • The Bone can’t write to the eMMC if booted from the eMMC. Instead, we’d need to get an image booting that is running out of RAM. I suggest booting a ‘buildroot’ image. This, in my mind, is the number 1 challenge to the whole project… we simply need a way to boot over USB reliably from every operating system. After that, everything is downhill.

The ‘buildroot’ image I already have it as a FIT image with everything which exposes eMMC over USB as a mass storage device. Again provide support for Win/OS X and we are done with a lot of problems.

7

Thanks for the feedback Jason and Vlad!

On last thing, in one of my previous replies I proposed another idea which I had in mind for the project. I’d like to know your views about this. I’m pasting it here:

8

Thanks for the feedback Jason and Vlad!

On last thing, in one of my previous replies I proposed another idea which I had in mind for the project. I’d like to know your views about this. I’m pasting it here:

I also had this other idea for the project in mind:
Normally the Cloud9 IDE is served from the BB, so users have to be have their BBs connected to code on it. If we could integrate the IDE separately into the NW.js app, it would enable users to code without the BB connected and later quickly upload the code after connecting. (somewhat like having an offline IDE). I did some searching about the integration. It looks like integrating the entire IDE may be difficult, as there seem to be a large number of dependencies (like python and g++) and that would make the installation process more difficult on the users end (which is clearly a big no no). But maybe we could integrate only the essential features, or maybe separately implement file system access and some other small features on top of something like the Ace editor (http://ace.c9.io/), to have a light-weight custom IDE.
What is your take on this?

c9.io already hosts Cloud9 IDE on the cloud and they have a downloadable version in an nw.js app of their own. I don’t really want to compete with them or take on an IDE as the focus of our efforts.