First off, it's not a torture. In fact it is the most fun I have had in my whole life, creating something from scratch, learning a lot of different weird things, architecting a whole system all on my own wish. It was the best time of my computer life!
Challenges
Is writting an OS a challenging deal? I would be disingeneous or lying or a megalomaniac if I say not. There are a list of challenges:
- I have no idea where to start
- Apart from a year 3 course about the theory of operating systems, I have no idea how or where or what to do to even get started writing an OS. I know a bit about Linux and old bootloader but that's it.
- I do not know assembly
- I had a course on computer architecture where I learned a bit of ASM of I don't remember what architecture! I don't know anything about ASM of x86 or ARM.
- A design of such thing is hard to judge
- I have a pretty good idea of what it should look like: A merge of memory management system and file system, Linux styled, but I don't have a way to judge if a system design is good or bad!
- I have no friends
- Really? Is this even a challenge mentionable? Yes! I don't have friends to talk to, and I don't want to bore my girlfriend to death with all these OS work. So I have to bare this weight on my own. To be honest I did myself disservice by staying with my girlfriend ALL THE TIME and met no one :D
- What are all these special memory addresses and segments?
- x86 is a very old and history-filled architecture. A lot of things rely on convention which is very sparsely documented and very hard to debug. You don't know what hits you until you know.
- Why do I even start from scratch? Can I not just modify Linux?
- My knowledge of Linux at that time was also mediocre and cannot support me changing the kernel structure and implement such thing. I'm forced to make a new OS because of it. I'm still not able to make changes in Linux (but now it's because I'm hopelessly depressed...)
- What should I also implement to make it an OS?
- At what point should I call it a success? It obviously would not host anything or do anything productive, it would just serve as a proof-of-concept. So what is successful is entirely up to my definition. I still think it's an incomplete piece of crap but hey, it has user-space and can load programs!
- How do I plan out the project and split it into small, manageable chunks?
- This isn't something that I know, this isn't something that I can reasonably plan and time for. I still need some ways to plan for it so that I can finish it and not fail!
- Where do I even find these kind of non-volatile memory?
- The hardware just came out. It was non-existent in the market, or it's stupid expensive. There is no way I can get it.
- Uncertainty
- I just don't know what would happen.
In short, this is just not something my 22-year-old brain has seen.
Solutions
By the fact that I'm not a dropout I successfully graduated, I must have made something that can remotely be called an OS, and have stepped through all these difficulties.
- Creating myself a starting point
- Google searches. Lots and lots of them. Reading and accumulating information on the topic of OS development day and night for months.
- Bootstrapping my development environment
- Gives myself a starting point: OS Dev Bare Bones.
- Sets up a nice little development environment with QEMU emulation, that provides debuggable and rapidly reloadable VM, that also emulates non-volatile memory.
- Sets up the toolchain to cross compile my little OS to a freestanding x86 environment.
- Creates all these things on AWS, an EC2 instance, that is powerful enough to run everything. Oh and I also configured it to startup as quick as possible, and self shutdown 1 hour on no activity so that I don't burn through my money. Because I don't have a decent computer that can do everything.
- Did not intentionally learn assembly at all
- Oh I found out that the need of assembly in my project is quite minimal midway, and I never bothered learning ASM fully. I did struggle choosing Intel or AT&T style assembly. I don't quite remember which way I went as it doesn't even matter in later stages.
- Did not judge the design at all, just work something out and show
- I just made something that makes sense to me and everyone that I asked. As a proof-of-concept I don't have any specific requirement apart from THE DESIGN OF AN UNIFIED DATA MANAGEMENT SYSTEM. Oh what the hell was I thinking not pondering on the design at least a million times!!!
- Anyway I just go with what makes sense. Having something done is better than talking about it everyday and not have something done.
- I still have no friend
- Yeah no one would listen to how I wrote an OS. That's also why I'm writing this blog like 5 years later. I held up, nothing tragic happened.
- But I know why I'm doing it. My motivation is clear: to explore how things can be done in OSes if such thing as non-volatile memory is real. I'm happy every day working on it, unlike some time later in my life.
- Did I mention that I did a lot of searches?
- Answered a lot of questions when I searched a lot on x86 systems. Learned a lot about TSS, GDT, LDT, x86 flags, CPU memory protection hierarchy, caching, system exclusive instructions, rings, memory-mapped area, low/high memory......
- The year of doing this OS project gave me much more than my 16 year formal education.
- I still don't know Linux
- I just give up on the idea of modifying Linux on day one. It is just too hard. Knowing what I don't know was a blessing. If I was to modify Linux I would not have finished the project nor graduated.
- Bare minimum is what I implemented
- Remember how we define an OS? It's a piece of software that talks to hardware on behalf of applications so that dumb things don't happen. That's exactly what I implemented, nothing more. Not even task switching, scheduling, CPU control, security,.....
- I drew a component map, and I tackled them one by one
- It's just good old divide-and-conquer. Also I don't know why no one is actually planning out a project in real world like this. Are project managers doing their job?
- Emulation, best way to do something you cannot do
- I did try to buy that Optane memory, but it did not happen until 2 years later.
- So for the entire project I just relied on file-backed memory on QEMU emulation.
- If it works it works.
- Just deal with it
- It was so fun I didn't even care about the risk of not being able to graduate!
No comments:
Post a Comment