Proposal for Cross platform USB boot

2.1 What is the name of your project?
Cross platform USB boot

2.2 What is the timeline for development of your project?
The Summer of Code work period is about 11 weeks long; tell us what you will be working on each week.

22 April 18 May - Fork vvu’s repo with his last year code and start rewriting it in a clean manner. The code should make us bring U-Boot on the board via USB booting.

19 May 24 May - Port the existing code to Windows and test it’s functionality.

25 May 3 June - Test dfu-util on both Linux and Windows if it works with u-boot DFU. Make u-boot scripts that on autorun run dfu-commands and wait for host to flash.

4 June 14 June - Start working on the GUI will be done in QT. Basic GUI will be tested on both Windows and Linux. It will be an easy 5-click tool which allows to flash either SPL, U-Boot, Kernel, FileSystem or all together in an image file.

15 June 23 June - Implement basic flashing functionalities with the GUI. Will use dfu-util as the underlying flashing tool. Tom Rini confirmed that dfu-util plays nice with U-Boot

24 June 1 July - Wrap up GUI design and flashing functionalities. Add option to backup the eMMC/uSD to host computer. Test code on both Win/Linux.

2 July 10 July - Implement a way that users can create their custom images with different parts SPL, U-Boot, Kernel, Filesystem. This would be super useful for people that need to have things already on the filesystem when they power up the board.

11 July 18 July - Test the above written code. Implement in the GUI a way for people to have links to predefined images and flash them. Will provide links for all major distributions for the BBB.

19 July 29 July - Write a codeless kext and test the code on a Mac. vvu told me he will help here with access on his Mac. I do not own own and this can be a bit problematic.

30 July 6 August - Getting the project to work on Mac too. I put more time here because development will be kinda slow.

7 August 18 August - Documenting and commenting code. Put up a page on elinux.org to have it available in a place where people frequently go and ask for help.

2.3 Describe your project in 10-20 sentences. What are you making? For whom are you making it, and why do they need it? What technologies (programming languages, etc.) will you be using?

My project consits in offering a way to flash your BeagleBone Black/White from Linux/Windows and Mac. The tool will have 2 main sections : boot and flash / boot. Flash and boot will flash the eMMC or uSD using an image supplied by the user or construct one with custom SPL, U-Boot, etc from the user. The boot option should just transfer everything to RAM and run from there. Here the issue is that the images can have a maximmum size of 512 MB.

To get the BBB into DFU mode the procedure will be as following:
The ROM Boot will expose a RNDIS interface to the host computer. The host will then give an IP address via BOOTP to the ROM. Afterwards the host can start sending via TFTP the SPL. Once the SPL runs it applies the same procedure as the ROM Boot to receive the U-Boot binary.
Once we have U-Boot on the board it will run some predefined script that will run DFU mode and expose the storage devices.

At this point the BBB is in DFU mode so we can write to it a full image or just parts of it like SPL, U-Boot, Kernel, Filesystem.

The tool will offer a way to download latest images automatically from bb.org site. Also I will include download links to major distributions of OS-es that run on BBB.

All programming will be done in C. The GUI for both Windows and Linux will be implemented in QT, since the main C library for USB communication will be libusb which is cross-platform compatible and is supported on both Linux and Windows, the code for one platform can be easily ported to another. Since both libusb and QT are supported by OS X, I will be able to test my software on a Mac without acquiring the machine, which I don’t posses, for a long time. On a Mac as vvu told me we need to have a codeless kext in order to prevent the kernel to acquire our device. Vlad told me that he tried and write such kext but did not finish it. Me and him will try at the end of the GSoC to port the project there too.

The end software can be used by anyone who requires often flashing and hence is in need of a fast reliable and easy method to do it. The user can start from pupils and student who might use this for study purposes and will end with professionals who can use it for industrial purposes of testing the hardware or loading the target image. The software can be used for testing specific functionalities. It can be implemented in a test bench.

As it was said before the GUI will be there simple, here is an example:
GUI

Where the user can pick to either flash or boot. The parts of the image can be targeted by selecting either a folder from the computer or provide a link, or one of the existing defaults that will be downloaded and after flashed\booted.

2.4 Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you described.

Since I have already very good knowledge in C programming and above basic knowledge in USB communication as well as network skills, the whole project will resolve to less time spent on research and more on coding.

None of my previous project can be related to this one in terms of content, but the fact that I complete all of them with success is relatable. The courses that I take now - Operating Systems and Operating System Lab have a good insight on the low level C programming and since I perform there with a high outcome, I am sure I will manage this project as well. I have knowledge of building GUIs in QT framework as well as Windows Forms, hence everything will boil down just to the matter of design.

I am sure that I will be able to complete the project in this timeline; time management, self learning and current experience are the key criteria here and I posses the needed skills. I am planning to set for myself earlier deadlines then those that are mentioned in the timetable so that I can be sure that I will keep with the schedule, and manage everything in time.

3 You and the community

3.1 If your project is successfully completed, what will its impact be on the BeagleBoard.org community?

The community will have a new (easier and faster) option to flash their images on to the board. I am sure that the ease of flashing the kernel on to the board might attract new people as well. Since the time of flashing the kernel will decrease drastic, new uses of the boards might emerge, this again will work towards an enlargement and is a general benefit for the community.

The fact that the project focuses on multi-platfrotming will give the community more freedom to work on their projects, hence not forcing them to stick to one or another platform. The community will be able to use their favourite software for coding and as a result leading to a better development experience.

Since during u-boot DFU will be used this will make the software flexible and hence give the end user more freedom in what he can do. For example the user will be available to ship only specific software peace he wants to run on the board while leaving the kernel unaltered. The use of DFU will add one more property and mainly flashing images greater then 512MB.

3.2 What will you do if you get stuck on your project and your mentor is not around?

First, eLinux.org and the BeagleBoard community wiki pages can be of help for the problems I might face as well as community blogs. BeagleBoards git repository might be of help as well (for example the u-boot repository). Since the community has both a mailing list and an IRC channel, those will come handy in case I can not find the solution on my own. But those for sure will not be the only resources I’d try to use, internet is wide, every problem has a solution, it just needs the right approach. In case the problem is not related to the board, I am sure I don’t have to explain in this answer how I will google the problem.

Link to melange as well: http://www.google-melange.com/gsoc/proposal/public/google/gsoc2014/abarbarosie/5629499534213120

Any feedback is appreciated.

2.1 What is the name of your project?
Cross platform USB boot

2.2 What is the timeline for development of your project?
The Summer of Code work period is about 11 weeks long; tell us what you will be working on each week.

22 April 18 May - Fork vvu’s repo with his last year code and start rewriting it in a clean manner. The code should make us bring U-Boot on the board via USB booting.

19 May 24 May - Port the existing code to Windows and test it’s functionality.

25 May 3 June - Test dfu-util on both Linux and Windows if it works with u-boot DFU. Make u-boot scripts that on autorun run dfu-commands and wait for host to flash.

Can you tie these milestones into the architecture description you give down below? It isn’t clear to me you understand what a “u-boot script that on autorun run dfu-commands” is. I have written some u-boot code and don’t know exactly what you mean here. Keep the language in the timeline at a high level (ie., this is the functionality of the project at this point in time) and break down the architectural details below.

I continue to be disappointed that there isn’t a plan for Mac. If Mac isn’t achievable, can you clearly research and document why?

4 June 14 June - Start working on the GUI will be done in QT. Basic GUI will be tested on both Windows and Linux. It will be an easy 5-click tool which allows to flash either SPL, U-Boot, Kernel, FileSystem or all together in an image file.

I think you aren’t clear on what each of these elements is and what role they play. Are you really talking about flashing each one separately or will the GUI simply allow you to replace each one in the boot process?

Perhaps I was understanding it wrong, but I thought the GUI allowed you to change each of the items loaded in the boot process.

Do I understand this right?

  • Load SPL over TFTP/RNDIS using ROM.
  • Load U-Boot over TFTP/RNDIS using SPL.

Choices:

  • Load image file to eMMC/uSD over DFU using u-boot.
  • Load kernel/ramdisk to RAM over (DFU?, TFTP/RNDIS?) using u-boot.
  • Load other binary application to RAM over (DFU?) using u-boot.

This doesn’t quite align with what has been written elsewhere, but it is what makes sense to me. I’ll leave it to you to work out why the existing comments don’t quite make sense to me.

15 June 23 June - Implement basic flashing functionalities with the GUI. Will use dfu-util as the underlying flashing tool. Tom Rini confirmed that dfu-util plays nice with U-Boot

When you tested dfu-util, wouldn’t you already have the flashing functionality working? Are their gaps in the functionality of u-boot when it comes to writing images to eMMC/uSD?

Will DFU provide individual file writes? Will it provide low-level (image) writes?

24 June 1 July - Wrap up GUI design and flashing functionalities. Add option to backup the eMMC/uSD to host computer. Test code on both Win/Linux.

2 July 10 July - Implement a way that users can create their custom images with different parts SPL, U-Boot, Kernel, Filesystem. This would be super useful for people that need to have things already on the filesystem when they power up the board.

I think you’ll want to implement having replaceable parts from the beginning. It is hard to rearchitect GUIs.

11 July 18 July - Test the above written code. Implement in the GUI a way for people to have links to predefined images and flash them. Will provide links for all major distributions for the BBB.

The design needs to consider this from the beginning. It is fine to delay implementation, but the UI should already consider getting a list of images and the image files from Latest Software Images - BeagleBoard.

19 July 29 July - Write a codeless kext and test the code on a Mac. vvu told me he will help here with access on his Mac. I do not own own and this can be a bit problematic.

30 July 6 August - Getting the project to work on Mac too. I put more time here because development will be kinda slow.

k. Now I see this mentioned. I’d just like a bit more info on it, but it can be reasonable to make it a “stretch goal” that might not be completed.

7 August 18 August - Documenting and commenting code. Put up a page on elinux.org to have it available in a place where people frequently go and ask for help.

I suggest having a user-guide on elinux.org AS-YOU-GO. We’ll need mentors and others to test well before you get to the final phase.

2.3 Describe your project in 10-20 sentences. What are you making? For whom are you making it, and why do they need it? What technologies (programming languages, etc.) will you be using?

My project consits in offering a way to flash your BeagleBone Black/White from Linux/Windows and Mac. The tool will have 2 main sections : boot and flash / boot. Flash and boot will flash the eMMC or uSD using an image supplied by the user or construct one with custom SPL, U-Boot, etc from the user. The boot option should just transfer everything to RAM and run from there. Here the issue is that the images can have a maximmum size of 512 MB.

To get the BBB into DFU mode the procedure will be as following:
The ROM Boot will expose a RNDIS interface to the host computer. The host will then give an IP address via BOOTP to the ROM. Afterwards the host can start sending via TFTP the SPL. Once the SPL runs it applies the same procedure as the ROM Boot to receive the U-Boot binary.
Once we have U-Boot on the board it will run some predefined script that will run DFU mode and expose the storage devices.

At this point the BBB is in DFU mode so we can write to it a full image or just parts of it like SPL, U-Boot, Kernel, Filesystem.

What if we just want to load a running bit of code into RAM? This procedure documentation doesn’t make it clear to me when we are loading to RAM vs. when we are writing to flash. Can you organize this section a bit more.

Some short answers:
DFU does provide flashing separate files, hence it will be implemented.
The user will be given the chose to customize both booting\flash and boot.
By DFU scripts I mean command that will set the target for flashing.

Modified proposal:

2.2 What is the timeline for development of your project?
The Summer of Code work period is about 11 weeks long; tell us what you will be working on each week.

22 April 18 May - Fork vvu’s repo with his last year code and start rewriting it in a clean manner. The code should make us bring U-Boot on the board via USB booting. Create a page on elinux.org; every following step involves paralel documentation of the work on the created page.

19 May 25 May - Port the existing code to Windows and test it’s functionality. Functionality: (Main functionality) bullet point of the project description.

26 May 5 June - Test dfu-util on both Linux and Windows if it works with u-boot DFU. Make u-boot scripts that on autorun apply dfu-commands. Description under (Flashing) bullet point of the project description.

6 June 17 June - Start working on the GUI will be done in QT. Basic GUI will be tested on both Windows and Linux. The user will given the chose to flash and boot\boot Boot descibed under (Boot) bullet points of the description, as well as the choice of eMMC\uSD as the target of the flashed image. It will be an easy 5-click tool GUI example under description section.

18 June 27 June - Implement basic flashing functionalities*(Main fuinctionality)* with the GUI. Will use dfu-util as the underlying flashing tool. Tom Rini confirmed that dfu-util plays nice with U-Boot.

28 June 6 July - Wrap up GUI design and flashing functionalities, integration. Add option to backup the eMMC/uSD to host computer. Test code on both Win/Linux.

7 July 16 July - Implement a way that users can create their custom images with different parts SPL, U-Boot, Kernel, Filesystem. This would be super useful for people that need to have things already on the filesystem when they power up the board.

17 July 25 July - Test the above written code. Implement in the GUI a way for people to have links to predefined images and flash them. Will provide links for all major distributions for the BBB.

26 July 29 July - Write a codeless kext and test the code on a Mac. vvu told me he will help here with access on his Mac. I do not own one and this can be a bit problematic.

30 July 10 August - Getting the project to work on Mac too. I put more time here because development will be kinda slow.

11 August 18 August - Documentation of the code.

2.3 Describe your project in 10-20 sentences. What are you making? For whom are you making it, and why do they need it? What technologies (programming languages, etc.) will you be using?

My project consits in offering a way to flash your BeagleBone Black/White from Linux/Windows and Mac. The tool will have 2 main sections : boot and flash / boot. Flash and boot will flash the eMMC or uSD using an image supplied by the user or construct one with custom SPL, U-Boot, etc from the user. The boot option should just transfer everything to RAM and run from there. Here the issue is that the images can have a maximmum size of 512 MB.

(Main functionality): To get the BBB into DFU mode the procedure will be as following:
The ROM Boot will expose a RNDIS interface to the host computer. The host will then give an IP address via BOOTP to the ROM. Afterwards the host can start sending via TFTP the SPL. Once the SPL runs it applies the same procedure as the ROM Boot to receive the U-Boot binary.
Once we have U-Boot on the board it will run some predefined script that will run DFU mode and expose the storage devices.

(Flashing): At this point the BBB is in DFU mode so we can write to it a full image or just parts of it like SPL, U-Boot, Kernel, Filesystem or separate files since DFU provides the option to flash files to rootfs. For this purpose I will run DFU scripts that will specify the are to which every part of the flashed imageparts of image\individual files has to be written.
(Boot): If the user wishes to boot the image rather flash it, we follow the same process as described under (Main functionality), but instead of going into DFU mode, proceed with sending kernel + ramdisk to the device as it was done in last years project.

The tool will offer a way to download latest images automatically from bb.org site. Also I will include download links to major distributions of OS-es that run on BBB.

All programming will be done in C. The GUI for both Windows and Linux will be implemented in QT, since the main C library for USB communication will be libusb which is cross-platform compatible and is supported on both Linux and Windows, the code for one platform can be easily ported to another. Since both libusb and QT are supported by OS X, I will be able to test my software on a Mac without acquiring the machine, which I don’t posses, for a long time. On a Mac as vvu told me we need to have a codeless kext in order to prevent the kernel to acquire our device. Vlad told me that he tried and write such kext but did not finish it. Me and him will try at the end of the GSoC to port the project there too.

The end software can be used by anyone who requires often flashing and hence is in need of a fast reliable and easy method to do it. The user can start from pupils and student who might use this for study purposes and will end with professionals who can use it for industrial purposes of testing the hardware or loading the target image. The software can be used for testing specific functionalities. It can be implemented in a test bench.

As said before, the GUI will be very simple:
GUI

Where the user can pick to either flash or boot. The parts of the image can be targeted by selecting either a folder from the computer or provide a link, or one of the existing defaults that will be downloaded and after flashed\booted.