OS161 LSP Support & Build on M1

This post was initially written as a piazza post for my OS course (CPEN 331), and since this course has a bad track record of saving the notes left behind by prior students I decided to post this here for posterity. It should be helpful for the standard distribution of OS161 as well, but I haven’t personally validated this.

Content

Part 1: LSP support

Are you using vscode/clion/helix/etc and are frustrated with lots of red squiggles everywhere that you can’t get rid of? Struggling to get your clearly superior terminal based editor to pickup symbols in this strange 22 year old codebase and consider using cscope a personal failure? Want nice lsp integration and can’t find anything online about how to set it up with os161? Hopefully this guide can help you on your path to editor satisfaction

If you’re using any editor worth its salt (xkcd#378) it should support LSP’s (what is a LSP?). For C the common two are clangd and ccls. I use clangd so the following solution will focus on support for clangd, installation instructions for clangd can be found here. A LSP server when combined with your editor should enable nice features like code completion, diagnostic errors, go-to-definition type navigation and more, they just need to figure out how to make sense of os161, which is understandably difficult for them by default as os161 has a relatively convoluted build process (BSD’s bmake, cross-compilation, multi thousand line bash config scripts etc.).

The key to setting it to work without pulling your hair out (for clangd atleast) is to use Bear.

Bear 🐻 its a tool that helps us create a compilation database for clangd, this tells clangd how each file in the os161 source tree will be built, which is enough information for clangd to then parse and understand our source code. It does this by doing computer crimes intercepting process executions during the build process (more cool info about how that is done here).

Once installed, it’s easiest to first clean your build (bmake clean where appropriate), then create a script that does all of your build steps (including ./configure !) then run something like bear -- ./build_everything.sh and it should create a compile_commands.json file with the captured build process info.

(Brief warning, bear seems to have some issues on mac, not sure why yet, but if the output is empty, current workaround is take the output from another system and just patch all of the file paths for your system manually. Also maybe intercept-build works? lmk in the comments)

Now that you hopefully have a compile_commands.json, how do you connect it and clangd to your editor? well that depends on your editor, for something like 🧬Helix, you just need your shiny new compile_commands.json file at the base of your directory, and clangd installed and it should just work™️. For vscode you can install the clangd extension then make sure to copy the compile_commands.json file to build/compile_commands.json, this required location is seemingly undocumented and unconfigurable… anywho. If you use clion these instructions should be helpful for getting it to use the compilation database, I think the rest should be easy to figure out if you use some other editor I haven’t listed, like neovim. Hopefully now developing your kernel will suck a bit less

Part 2: Building OS161… M1 Edition

While the provided instructions for installing os161 may help you get close to a working install on default ubuntu, for m1 the list of things you need to fix is rather long. Only some of the required patches are documented in piazza so I figured I would note down what it took for me to get it running with the hope that someone else may be horrified can have the pleasure of running os161 locally, rather than needing to rely on the ece servers or other remote machines.

So first the normal steps, which should be installing homebrew, and the requisite tools/libraries.

brew install bmake mpc gmp wget mpfr

then use homebrew to actually install gcc and then place gcc and c++ symlinks on your $PATH (preferebly in /opt/homebrew/bin/) pointing towards the brew installed gcc-13 and c++-13; This is nessicary since for reasons I don’t understand, MacOS will alias clang as gcc so the build scripts will throw clang specific errors if you try and use the default gcc command.

You will then also need to install a version of python2 for building gdb, this is painful at this point as the last major release for python2 (2.7) was over a decade ago, with lt now officialy EOL’d in 2020; meaning its not particularly easy to install anymore as you really shouldn’t be using it, but alas os161’s 20 year old code bases 17k long bash script configure file demands python2, so we must pay a price.

The easiest way to install python2.7 is with pyenv; you can use pyenv to install the last release of python2: 2.7.18, and then use pyenv to configure it as your global python2, you must do this by specifying both a version of python3 and python2 at the same time with something like pyenv global 3.11.4 2.7.18; ok great but your going to have another problem, the build script will find python2, but it won’t know where the headers are cause the script is jank python headers are hard to find…

So you will want to figure out where your python2 shim from pyenv tells the script it should look, this isn’t where they actually are because software is made for suffering i dont actually know, anywho this tells us where we need to put them, this looks something like

$ ~/.pyenv/versions/2.7.18/bin/python-config --libs
-lpython2.7 -lintl -ldl -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -framework CoreFoundation

ok so python2 says the library files are in /opt/homebrew/lib, great, where are the really? In ~/.pyenv/versions/2.7.18/lib/ of course, this just means we need to symlink the libpython2.7.dylib file in that library folder to the homebrew lib folder.

Ok, now to actually trying to start building the darn program

start with jades build script since its much nicer than the provided build scripts for this course.

You will need to implement tip #2 step 2&3 from the install tools on macos guide, its easy enough to just fix these after the build script fails, since Jades script doesn’t redownload, uncompress, or recompile things that have already been done, its easy to navigate to the source and just patch the files.

then hopefully you will hit compiling sys161, this is where jade left a nice note about the problem we are going to hit, the symbol extra_selecttime should be declared extern in $SYS161/include/onsel.h, but its not, and unfortunately the nice one line sed command in the script won’t work since it seems the default sed on macos isn’t very compatable with the format of other sed versions, meaning the command will fail and you will have to go and manually fix the file instead.

changing the line uint64_t extra_selecttime to extern uint64_t extra_selecttime, then bmake clean in $SYS161 then build again and it should work.

Then you should be good to go and compile the kernel and get things running!

I’m sure I’m still missing a few extra steps, but hopefully someone finds this helpful and typing this out wasn’t a huge waste of time