GSoC 2010 - Porting Minix 3 to Beagle Board

Hi ,
i am Tushar Dadlani a third year computer science student. I have
always been interested in the beagle board project, but never had the
opportunity to work on one. I have a very basic experience of running
minix on a VM and also done very basic ARM assembly in cross compiler
environment. What type of expertise are you looking for from an
applicant to this project ?

Regards,
Tushar Dadlani

Dear Tushar,

Thanks for your interest!

As I coined the project idea, let me try to give an answer to your question.

As far as I am informed some work has been done on this in the past.
Last activity I am aware of is someone who worked on this as part of
his polytechnic graduation project (seems to have ended last January).
I am not aware if any of this has been published, but I expect that it
can be made available.

Wrt required expertise:
Good experience in C programming is a must. Most of the programming
will be done in C.
Some assembler knowledge might come in handy. That also depends on the
state of what is already available.
Also it helps greatly to have some understanding on operating systems
and on microprocessor architecture (and especially ARM).

Note that you will be dealing with hardware at a very low level and in
order to have a decent understanding of what is going on it you should
know about interrupts, memory mapped i/o, memory management and
possibly DMA and board initialisation.

Wrt the project itself:
A first task (which should be executed a.s.a.p) is to find out what
the actual state of the art is wrt Minix/ARM. Guess google can help
here. Also there is a minix mailing list.
After that we need to identify what is needed to get this running.
Guess there are some low level issues to be resolved, but I expect
most of the work to be to actually write or port the needed device
drivers. TTY is probably going to be easy (as you might be able to
reuse a lot from the PC implementations as uarts seem to become fairly
standardized wrt interface). USB and SD might be somewhat more
complicated. Audio and video are probably going to be even harder as
there is probably little in the PC version that you can reuse or learn
from.

Personally (without knowing too much what there is now), I would be
inclined to call the process a success if you managed to get a booting
minix image with tty, sd, usb hub, usb mass storage and usb ethernet
drivers.

Does this help somewhat?

Best regards, Frans

Btw also saw your message on irc, but as it was around midnight here,
I was not online to answer it.

Quoting from irc:
(10:55:11 PM) tush726: The GSoC ideas page has an idea to port Minix 3
to Beagle Board which is effective porting Minix 3 to ARM if i am not
mistaken
(10:55:44 PM) tush726: There is already a Minix 3 project on porting
Minix 3 to ARM
(10:55:56 PM) tush726: which is not under GSoC
(10:56:11 PM) tush726: Won't both the projects clash

I doubt so. I'm not too sure about the progress on the non-gsoc project.
There has been a gsoc08 project on this, but I never saw results of that one.

The graduaton project I was referring to was done by Leon Melis.
From his linkedin page:
Worked on the MINIX3 ARM port as a "graduation project". I am
currently working part time on the project and I am planning to
release it as open source.

And this was a fairly recent post on the minix mailing list:
http://groups.google.com/group/minix3/browse_thread/thread/82f0e1c90d00482f/e34f5cc4d9b1d685?show_docid=e34f5cc4d9b1d685

Anyway, even if the other project is still alive, I think there is
enough work for a beagle project (if alone creating the needed device
drivers).

Frans

Morning,

Quick intro: I've been doing some bare-metal beagleboard stuff just
for entertainment, and I also had a quick look at minix 3 (and haiku)
along the way.

Wrt required expertise:
Good experience in C programming is a must. Most of the programming
will be done in C.

It's almost all C, so you need to know it well.

Note that you will be dealing with hardware at a very low level and in
order to have a decent understanding of what is going on it you should
know about interrupts, memory mapped i/o, memory management and
possibly DMA and board initialisation.

Definitely need DMA, but it's about simplest hardware in the machine.
Hopefully the rom and bootloader do most of the board init. I'd add
caches to that too.

Wrt the project itself:
After that we need to identify what is needed to get this running.
Guess there are some low level issues to be resolved, but I expect
most of the work to be to actually write or port the needed device
drivers. TTY is probably going to be easy (as you might be able to
reuse a lot from the PC implementations as uarts seem to become fairly
standardized wrt interface). USB and SD might be somewhat more
complicated. Audio and video are probably going to be even harder as
there is probably little in the PC version that you can reuse or learn
from.

TTY is trivial - hardware is the same as pc serial except memory
mapped. SD don't know about. Audio/Video are relatively simple since
the hardware is pretty simple, even if the manual isn't (for
dvi/s-video at any rate, not sure about lcds). Interrupts depend on
the ARM implementation, so any work done may not be usable.

USB will be the big thing - last time i looked minix3 had no usb
support - although I see a project released just a week ago on
googlecode for it, but it is uhci only.

Personally (without knowing too much what there is now), I would be
inclined to call the process a success if you managed to get a booting
minix image with tty, sd, usb hub, usb mass storage and usb ethernet
drivers.

Sounds about right to me from what I've looked at - oh and something
for managing the NAND, and maybe expansion stuff. Although given that
USB will be for the most part the same as any PC implementation, huge,
and they still don't have one, maybe it would be better to focus on
beagle-specific stuff at this point?

tty, nand, sd, expansion i/o (gpio/serial interfaces), neon, video,
audio in approximately that order is quite a lot to look at already.
And if that isn't enough, there's always the dsp. I mean obviously
USB is very important, but as an embedded device you can still do a
lot without it.

When I last looked (a few months ago), minix also suffers a lot from
it's pc architecture roots. There's no framework for multiple
architectures. And things are a bit over the place as students have
hacked in additional features such as virtual memory.. It's a far cry
from the original `book version' in terms of 'clean simple design'.

!Z

Dear Tushar,

Thanks for your interest!

As I coined the project idea, let me try to give an answer to your question.

As far as I am informed some work has been done on this in the past.
Last activity I am aware of is someone who worked on this as part of
his polytechnic graduation project (seems to have ended last January).
I am not aware if any of this has been published, but I expect that it
can be made available.

Thanks for that info.

Wrt required expertise:
Good experience in C programming is a must. Most of the programming
will be done in C.

Decent C programming experience.

Some assembler knowledge might come in handy. That also depends on the
state of what is already available.

As a fact was trying to understand the pc assembly of the boot process
already there in the minix sources. No experience in ARM assembly.
But can learn it if required.

Also it helps greatly to have some understanding on operating systems
and on microprocessor architecture (and especially ARM).

Have a decent understanding of Operating Systems and been studying
about the Linux Kernel since quite some time.

Note that you will be dealing with hardware at a very low level and in
order to have a decent understanding of what is going on it you should
know about interrupts, memory mapped i/o, memory management and
possibly DMA and board initialisation.

I realised the low level requirement of this project.

Wrt the project itself:
A first task (which should be executed a.s.a.p) is to find out what
the actual state of the art is wrt Minix/ARM. Guess google can help
here. Also there is a minix mailing list.

Could you clarify what do you mean by the state of the art wrt Minix/
ARM .

After that we need to identify what is needed to get this running.
Guess there are some low level issues to be resolved, but I expect
most of the work to be to actually write or port the needed device
drivers. TTY is probably going to be easy (as you might be able to
reuse a lot from the PC implementations as uarts seem to become fairly
standardized wrt interface). USB and SD might be somewhat more
complicated. Audio and video are probably going to be even harder as
there is probably little in the PC version that you can reuse or learn
from.

Personally (without knowing too much what there is now), I would be
inclined to call the process a success if you managed to get a booting
minix image with tty, sd, usb hub, usb mass storage and usb ethernet
drivers.

Does this help somewhat?

Thanks a lot for all the information .

Best regards, Frans

What sort of deliverables are you looking at by the end of the
summer?

Regards,
Tushar Dadlani

[...]

Wrt the project itself:
A first task (which should be executed a.s.a.p) is to find out what
the actual state of the art is wrt Minix/ARM. Guess google can help
here. Also there is a minix mailing list.

Could you clarify what do you mean by the state of the art wrt Minix/
ARM .

As explained in my 2nd email in this thread, some work has already been done.
Instead of starting from scratch, I would suggest investigating first
what has been done already and whether that could be used as a
starting point (and hence save some time).

[...]

What sort of deliverables are you looking at by the end of the
summer?

I'd say the minimum set of deliverables is:
- a minix image suitable to be flashed onto beagle (or to be run from SD).
- the image should contain a filesystem, a tty driver and a shell (so
users can access it over serial and do something on it).
- instructions to generate this image (including setting up a cross
environemt), so others can reproduce this. Personally I would have a
slight preference to have a generation process that uses
openembedded/bitbake, but other options are also possible.

Perhaps a good acceptance test would be: can a user play a game of
nethack on the system. (nethack uses the terminal and the filesystem,
is already available for minix PC and, apart from curses does not seem
to have much externel dependencies.

Best regards, Frans

> What sort of deliverables are you looking at by the end of the
> summer?

I'd say the minimum set of deliverables is:
- a minix image suitable to be flashed onto beagle (or to be run from SD).
- the image should contain a filesystem, a tty driver and a shell (so
users can access it over serial and do something on it).
- instructions to generate this image (including setting up a cross
environemt), so others can reproduce this. Personally I would have a
slight preference to have a generation process that uses
openembedded/bitbake, but other options are also possible

Perhaps a good acceptance test would be: can a user play a game of
nethack on the system. (nethack uses the terminal and the filesystem,
is already available for minix PC and, apart from curses does not seem
to have much externel dependencies.

Best regards, Frans

I have started working on my application and I wanted to know about
what everyone over here feels about the fact , if the project
mentioned in this thread is successfully completed , what will its
impact be on the BeagleBoard.org community?

Thanks, Tushar

- instructions to generate this image (including setting up a cross
environemt), so others can reproduce this. Personally I would have a
slight preference to have a generation process that uses
openembedded/bitbake, but other options are also possible.

Wouldn't the ultimate goal be to merge with minix3 upstream?
oe/bitbake is really out of scope for that isn't it? A lone fork
would probably just wither and die.

Perhaps a good acceptance test would be: can a user play a game of
nethack on the system. (nethack uses the terminal and the filesystem,
is already available for minix PC and, apart from curses does not seem
to have much externel dependencies.

and/or edit and compile a hello world from the console?

- instructions to generate this image (including setting up a cross
environemt), so others can reproduce this. Personally I would have a
slight preference to have a generation process that uses
openembedded/bitbake, but other options are also possible.

Wouldn't the ultimate goal be to merge with minix3 upstream?
oe/bitbake is really out of scope for that isn't it? A lone fork
would probably just wither and die.

You definitely have a point here.

Some thoughts:
I feel merging with upstream is done much more easily when we have
something running.
As minix currently is only for x86 host and x86 target, I am not too
sure how well-separated things are.
Anyway it seems wise not to deviate too much from the structure that
is currently there.

For BB I see two options:
- use a cross compiler on x86. Openembedded already has all of the
infrastructure present, so we could leverage from that (and the
required bitbake file would probably be quite small).
- use a native compiler on BB (e.g. under angstrom, debian or ubuntu).
This is closer to what exists on x68. Disadvantage is a longer
development cycle. as the BB is slower. Also as I can imagine other
ARM offsprings (e.g. for hawkboard) native compilation would be more
troublesome on those platforms.

Actually it would even be possible to start with the 2nd option and
eventually move to the 1st option.

Perhaps a good acceptance test would be: can a user play a game of
nethack on the system. (nethack uses the terminal and the filesystem,
is already available for minix PC and, apart from curses does not seem
to have much externel dependencies.

and/or edit and compile a hello world from the console?

That would indeed also be a good acceptance test. I expect getting gcc
to run on the target will be more complicated than getting nethack to
run, as i expect gcc has more dependencies on external packages. But
yes, I would consider a native compilation of hello world also to be a
good acceptance criteria.

I'm a (part-time) teacher IRL, and generally I'd like to give students
a lot of freedom in the way they tackle their projects (especially
graduation projects, and I feel this should be on-par with a
graduation project). I haven't done GSOC projects before, but my idea
was to handle this the same way.

My suggestion is that interested students write down themselves
- what they want to achieve
- a rough planning for the project
- what would be the criteria for success (and this could be graded
into a few levels)
- and maybe also a little bit on how they want to achieve it.

Especially the order of activities and how to achieve it could be interesting.
One of the issues in this project is how to start the initial kernel.
For now I see a few alternatives:
- directly from MLO (makes it a bare metal app)
- from u-boot (nice because u-boot will already do some hardware
initialisations, not sure if this will need additional work though)
- load the kernel with JTAG (e.g. openOCD). Of course not a viable
option in the end, but could be nice as a starter
- use linux/kexecboot as a bootstrap
This is again something to be decided by the student. (actually making
an analysis of this with pro's and con's of each alternative would
make a nice qualification task).

One final thing:
I'm more than happy to mentor this project. I have a large amount of
embedded software and operating systems experience and a lot of
Minix-1 experience.
Minix-3 is quite new for me too though. I will be available all
summer, but due to some personal constraints, I won't be available
from april 9 to may 3, I expect to have hardly to no time and hardly
to no email access during that period.

Best regards,
Frans

For BB I see two options:
- use a cross compiler on x86. Openembedded already has all of the
infrastructure present, so we could leverage from that (and the
required bitbake file would probably be quite small).
- use a native compiler on BB (e.g. under angstrom, debian or ubuntu).
This is closer to what exists on x68. Disadvantage is a longer
development cycle. as the BB is slower. Also as I can imagine other
ARM offsprings (e.g. for hawkboard) native compilation would be more
troublesome on those platforms.

Not to mention it's a bit hard to test the hardware if you're already
using it. Once past the problem of just building it, it will help not
to have to reboot twice just to test something out.

and/or edit and compile a hello world from the console?

That would indeed also be a good acceptance test. I expect getting gcc
to run on the target will be more complicated than getting nethack to
run, as i expect gcc has more dependencies on external packages. But
yes, I would consider a native compilation of hello world also to be a
good acceptance criteria.

I thought minix had gcc, but it looks like they have their own
compiler, so it's trickier than i thought.

One final thing:
I'm more than happy to mentor this project. I have a large amount of
embedded software and operating systems experience and a lot of
Minix-1 experience.

I think Jason is looking for 2xmentors for each project, so I'll put
my hand up as a tentative here too if it gets that far. Just about
all the projects outside of linux/android (blah) and hardware (alas,
lack of experience/equipment) look interesting to me mind you.

Michael

For BB I see two options:
- use a cross compiler on x86. Openembedded already has all of the
infrastructure present, so we could leverage from that (and the
required bitbake file would probably be quite small).
- use a native compiler on BB (e.g. under angstrom, debian or ubuntu).
This is closer to what exists on x68. Disadvantage is a longer
development cycle. as the BB is slower. Also as I can imagine other
ARM offsprings (e.g. for hawkboard) native compilation would be more
troublesome on those platforms.

Not to mention it's a bit hard to test the hardware if you're already
using it. Once past the problem of just building it, it will help not
to have to reboot twice just to test something out.

I must be getting old :slight_smile:
A lot of the minix 1.5 work for Atari ST was done exactly as you
describe. I think I already had a hard disk at that time :slight_smile:
It is definitely doable, and it greatly helps to think before you
code. Also it helps to devise good tests in case there is a problem.

Btw the dual boot problem could also be resolved by having two boards
(one for working on, one for testing). Actually the development one
could even be a different ARM board or even a QEMU setup.

Having said that my initial preference would be to use a cross compiler setup.

and/or edit and compile a hello world from the console?

That would indeed also be a good acceptance test. I expect getting gcc
to run on the target will be more complicated than getting nethack to
run, as i expect gcc has more dependencies on external packages. But
yes, I would consider a native compilation of hello world also to be a
good acceptance criteria.

I thought minix had gcc, but it looks like they have their own
compiler, so it's trickier than i thought.

Afaik they support both gcc and their native compiler (which is based
upon the ACK, the amsterdam compiler kit).
I'm pretty sure their native compiler will support ARM. Last time I
looked (which was a looooong time ago) it already had backends for
various architectures.

One final thing:
I'm more than happy to mentor this project. I have a large amount of
embedded software and operating systems experience and a lot of
Minix-1 experience.

I think Jason is looking for 2xmentors for each project, so I'll put
my hand up as a tentative here too if it gets that far. Just about
all the projects outside of linux/android (blah) and hardware (alas,
lack of experience/equipment) look interesting to me mind you.

Thanks alot.

Guess it is now up to Tushar to give this some momentum.

Have fun!
Frans

Thanks a lot for all the information. Not got any reply from the minix
mailing list about the ARM support as yet. I also don't own a beagle
board for testing which I would mention on my student proposal.

Regards,
Tushar

The latter is an issue that can be resolved if your application passes.
(at least that is what I understood from Jason's messages).

Frans

Frans/Tushar - Yes, definately, beagleboard.org will supply the board
and cables for every approved project.

Cathy

Contacted someone at the Amsterdam Free University on this. Got this
as a reply. Seems this is a can of worms which is probably too
complicated for GSOC.
(btw soemone also tried it as gsoc project in 2008)

Frans.

=== start quote

some people have tried to port Minix3 to ARM. Nobody ever succeeded.
As I said in the email you are quoting, Minix3 is so ia32 specific
that porting it anything is impossible without rewriting major parts
of the code. The problem is not the ARM specific assebly and such, it
is mostly about memory management. I've told this the guy who tried
that tast time ... well after several month of strugling he came to
the same conclussion :frowning: Btw. he originaly tried to port 3.1.3 which
does not even use paging which indeed is a problem on ARM. So I would
say that everybody should stop thinking about porting it to ARM before
the virtual memory management is fixed. Imho x86-64 is the only
possible port right now and even this is not straigt forward as x86-64
does not use segmentation except %gs and %fs.

Hi ,
Okay. I wanted to know , I also had a look at port of plan 9 to the
Beagle Board on the Plan 9 GSoC ideas. But as it is being mentored by
someone from Plan 9 community they require me to have a Beagle Board
for testing and other related activities. I have worked on plan 9 and
have also tried out glendix. Would you be interested in mentoring that
project alongside the mentor from Plan 9 ? Haven't yet got any reply
from that mentor regarding the same. Had mailed him a few days back.

Should I consider applying for that project as a part of Beagle Board
GSoC ? Just had a look at this thread on the plan 9 mailing list .
http://groups.google.com/group/comp.os.plan9/browse_thread/thread/3415f0c169dfc2dd?pli=1

Thanks for your insight well before hand.

Regards,
Tushar Dadlani

Frans,

Well that's a bit of a pity, but better to know rather than fail
catastrophically. I remember something about the memory management
being a particular mess, but I forgot about all that segment stuff.

Michael

I'd say go ahead and propose it.
As far as mentoring concerns: I have no problems to mentor you
(provided you're assigned to me and you feel that is a good plan too).
Note though that I have zero knowledge on plan 9 (although I have a
fair amount of knowledge when it comes to operating systems and their
implementation in general).

Frans.

I agree. I suggest trying to find a second mentor either in your time
zone or as familiar with what you are trying to do as possible.
http://tinyurl.com/bbgsoc should tell you some other possible mentors
to contact.

Well i'd be interested if you need a `beagleboard co-mentor'.
Although I also have zero knowledge about plan 9 too. I didn't pip up
because I presume there'll be need of a plan 9 mentor, and i'm not
sure how it should go.

Tushar, If you're going to do it, just get the proposal in asap,
otherwise there's no time for updating it based on reviews. The
deadline is nearly here isn't it?

Michael