LKOS2 — A Linux Kernel for OS/2 (project announcement, early alpha)

[Rich Walsh]

it was a constant battle to keep Claude from hallucinating facts about OS/2's internals and going off the rails.

I asked Claude Code about this. Here's a snippet from its response:

Q: Is there anything I can do to enhance Claude's understanding of OS/2 and its API? The goal is a long-term familiarity with it across multiple projects and possibly multiple users.

A: My baseline OS/2 knowledge is real but shallow and unreliable in exactly the way that hurts porting work: I'll confidently produce a `DosOpen` call with plausible-but-wrong flag names, or invent an `OS2.H` API that never existed. The fix isn't prompting me to "know OS/2 better"; it's putting authoritative material on disk where I can grep it, plus a small set of rules about what I must never guess.

The single biggest upgrade is the OS/2 Toolkit headers checked into a repo the agent can read. [...] Then a `CLAUDE.md` rule roughly: *never write an OS/2 API call without confirming its prototype in `refs/toolkit/include`; if it's not there, say so instead of inventing it.* That one rule kills most of the failure mode.

[phaelonimaire]

Hello Phaelonimaire,

Is your project associated with this GitHub repository in any manner?
https://github.com/hiroshiyui/warpine

Regards,
Alex

Not sure.  I wasn't aware of the project until now.  From my brief scan of it, they're taking a Wine/2ine approach, where they provide DLL interfaces to run OS/2 programs on Linux.  My approach is hilariously (and more crazily) upside down, where I provide a low-level shim to allow the original IBM OS/2 DLLs to work.  The code I looked at doesn't seem to have the traditional input chain (KBD$/PMDD.SYS(SINGLEQ$)), instead replacing that DLL-side with hooks to whatever linux backend.  There's probably some synergy between the projects, but the ideology seems upside-down from mine, not to mention C VS Rust. 

[phaelonimaire]

Hello Roderick

Is it an option to use netlabs.org and create a project on there ?

I think it is a good option to use github like Bitwise Works is doing now. The netlabs server sometimes works and sometime don't.

Regards

I have a github set up, I just haven't put anything there yet.  I will soon.

[phaelonimaire]

it was a constant battle to keep Claude from hallucinating facts about OS/2's internals and going off the rails.

I asked Claude Code about this. Here's a snippet from its response:

Q: Is there anything I can do to enhance Claude's understanding of OS/2 and its API? The goal is a long-term familiarity with it across multiple projects and possibly multiple users.

A: My baseline OS/2 knowledge is real but shallow and unreliable in exactly the way that hurts porting work: I'll confidently produce a `DosOpen` call with plausible-but-wrong flag names, or invent an `OS2.H` API that never existed. The fix isn't prompting me to "know OS/2 better"; it's putting authoritative material on disk where I can grep it, plus a small set of rules about what I must never guess.

The single biggest upgrade is the OS/2 Toolkit headers checked into a repo the agent can read. [...] Then a `CLAUDE.md` rule roughly: *never write an OS/2 API call without confirming its prototype in `refs/toolkit/include`; if it's not there, say so instead of inventing it.* That one rule kills most of the failure mode.

What I did last weekend was amass all of the documentation I could find and built a claude-friendly Claude toolkit for OS/2.  It contains the information on various OS/2 APIs, as well as some of the tooling I've made.  My goal is to push this out to my github as something one could point claude at as "I want to to X OS/2 project, here's a toolkit/knowledgebase" and it work. 

[Dariusz Piatkowski]

...What I did last weekend was amass all of the documentation I could find and built a claude-friendly Claude toolkit for OS/2.  It contains the information on various OS/2 APIs, as well as some of the tooling I've made.  My goal is to push this out to my github as something one could point claude at as "I want to to X OS/2 project, here's a toolkit/knowledgebase" and it work...

         ^
         ^
This HERE is huge in terms of potential outcome this could bring about!

Consider the almost standard challenge/gap we in this tiny OS/2 group of ours face, that being: the in-depth technical expertise that we can easily access to answer standard development type inquiries.

It is NOT that we do not have folks on here that have this skillset, rather there is only so much time that any of us can devote to answering questions.

Ultimately that means any learning we try to do is significantly slowed down, and in some cases only gets to a dead-end.

Heck, I'm stuck in my DISKIO enhancement process b/c I cannot figure out how to establish a mapping between a logical drive name (so C:, D:, etc..) and the physical device this drive exists on (b/c DISKIO is going to test the physical device, but from a human interface perspectice we mostly work with drive names, so DISKIO should have the capability to accept that as an input of a device to be tested). Seems simple, like the most basic of all things, but AFAIK there isn't an API that actually exposes this and so one must hack through multiple other APIs to try to get to that answer. Discouraging at best...

Sooo...if there is even a remote possibility to build a OS/2 API Virtual Guru/Advisor that can do some of that grunt/digging work for me...oh my, I would sign up for that in a flash!!!

BTW - I say this from a point of having parsed ALL OS/2 toolkit header files and built the Visual SlickEdit reference files so that if you do happen to use VSE to code with, you actually get some semblance of API awareness in your IDE and things start to feel a little less and less like fishing for a needle in a haystack! Now if we could super-charge that with a call to an AI agent => BAM!!!! ...that would be a massive improvement!

[Digi]

• More memory — programs keep the familiar OS/2 arena limits, but the old 4 GB system-wide ceiling is gone: every OS/2 process gets its own memory, bounded only by installed RAM.

What does the amount of installed RAM have to do with the address space available to software?

[Dave Yeo]

Each process or session could have its own 4GB (actually 3.5 or less) address space.
Even today, multiple DOS sessions each have its own address space and if you load half a dozen DOS sessions each using the maximum 1GB, we'd run out of memory (ram) and swap space.
Whether this would work remains to be seen. Things like shared memory may make it hard.

[Digi]

I would also like to propose a discussion on the topic of abandoning 16-bit code wherever possible. I understand that for many OS/2 users DOS and WinOS2 support is a sacred cow and almost a religious issue. :-) But let's look at this soberly - for DOS there is DOSBOX, winos2... what programs are there for win16 that don't have modern equivalents that can be run on separate real Windows systems? 16-bit code has no future, CPUs will not support it for long. Getting rid of it completely is a separate issue. I understand that won't be easy. There will be problems with drivers...

Let's at least start discussing this.

PS: I have been using the PROTECTONLY=YES mode on all my systems for many years.

[phaelonimaire]

...What I did last weekend was amass all of the documentation I could find and built a claude-friendly Claude toolkit for OS/2.  It contains the information on various OS/2 APIs, as well as some of the tooling I've made.  My goal is to push this out to my github as something one could point claude at as "I want to to X OS/2 project, here's a toolkit/knowledgebase" and it work...

         ^
         ^
This HERE is huge in terms of potential outcome this could bring about!

Consider the almost standard challenge/gap we in this tiny OS/2 group of ours face, that being: the in-depth technical expertise that we can easily access to answer standard development type inquiries.

It is NOT that we do not have folks on here that have this skillset, rather there is only so much time that any of us can devote to answering questions.

Ultimately that means any learning we try to do is significantly slowed down, and in some cases only gets to a dead-end.

Heck, I'm stuck in my DISKIO enhancement process b/c I cannot figure out how to establish a mapping between a logical drive name (so C:, D:, etc..) and the physical device this drive exists on (b/c DISKIO is going to test the physical device, but from a human interface perspectice we mostly work with drive names, so DISKIO should have the capability to accept that as an input of a device to be tested). Seems simple, like the most basic of all things, but AFAIK there isn't an API that actually exposes this and so one must hack through multiple other APIs to try to get to that answer. Discouraging at best...

Sooo...if there is even a remote possibility to build a OS/2 API Virtual Guru/Advisor that can do some of that grunt/digging work for me...oh my, I would sign up for that in a flash!!!

BTW - I say this from a point of having parsed ALL OS/2 toolkit header files and built the Visual SlickEdit reference files so that if you do happen to use VSE to code with, you actually get some semblance of API awareness in your IDE and things start to feel a little less and less like fishing for a needle in a haystack! Now if we could super-charge that with a call to an AI agent => BAM!!!! ...that would be a massive improvement!

Through claude, this has been the only real way to make application development reliable for OS/2.  As you said, finding documentation (when it exists at all) is difficult, and you can never be sure if it's hard to find because you haven't found the correct reference, or it was just never documented. 
Using what I made so far, I ported Windows Notepad2 last night to OS/2 pretty much end-to-end.  I'll be making that available too. 

[phaelonimaire]

• More memory — programs keep the familiar OS/2 arena limits, but the old 4 GB system-wide ceiling is gone: every OS/2 process gets its own memory, bounded only by installed RAM.

What does the amount of installed RAM have to do with the address space available to software?

Fair question — my wording was sloppy.

As you may already know, installed RAM has nothing to do with how much address space a program gets. That part doesn't change at all. An OS/2 program is still a 32-bit program with the same arena limits it's always had: same 512MB line, same VIRTUALADDRESSLIMIT, same ~3GB ceiling at best. None of that moves, and none of it was meant to.

What changes is how much RAM the *machine* can use.   The old kernel can only see about 4GB of RAM, total. That isn't a per-program limit — it's the kernel's own limit on how much memory it can keep track of. It can't count any higher than that. Put 32GB in the box and OS2KRNL uses 4GB and ignores the rest.   
 
A 64-bit kernel doesn't have that problem. It can use every gigabyte you install. Each OS/2 program still gets exactly the same 32-bit view and limitation it always had — but the kernel is able to address the total physical RAM installed. When a new 32-bit OS/2 process is created, the Linux kernel can back it with physical RAM from anywhere in the machine, including the memory above the 4GB mark that OS2KRNL could never reach on its own (32-bit physical address limitation). The new process still only ever sees its own 32-bit address space — it just no longer matters where in physical RAM that address space actually lives, because mapping it is the kernel's job, not the program's.

If you ran 32-bit Linux or Windows with PAE back in the day, same idea: each program still stuck with its usual couple of GB, but the machine could hold 64GB
 and keep a lot of programs running in it at once.

One thing that does NOT improve: the shared area where DLLs and named shared memory live. Still one 512MB region for the whole system, same as always —  that's what keeps shared memory behaving the way OS/2 programs expect. Extra RAM buys you nothing there.

Also of note, I'm trying to do a faithful reimplementation of the OS2KRNL's global shared memory right now because I don't know what breaks if it changes.  When things are stable, we might be able to remove some of these traditional limitations.

So "bounded only by installed RAM" meant the machine's usable memory, not any one program's reach. I should have said that.

[phaelonimaire]

I would also like to propose a discussion on the topic of abandoning 16-bit code wherever possible. I understand that for many OS/2 users DOS and WinOS2 support is a sacred cow and almost a religious issue. :-) But let's look at this soberly - for DOS there is DOSBOX, winos2... what programs are there for win16 that don't have modern equivalents that can be run on separate real Windows systems? 16-bit code has no future, CPUs will not support it for long. Getting rid of it completely is a separate issue. I understand that won't be easy. There will be problems with drivers...

Let's at least start discussing this.

PS: I have been using the PROTECTONLY=YES mode on all my systems for many years.

Unfortunately, it's not merely a *sacred religious cow*; 16-bit code is *everywhere*. It's all throughout VIO, PM, DASD, drivers, and the very way OS/2 handles memory. It's not a clean break at any level - it's all mangled together in a mess. VIO, KBD and MOU don't even have a 32-bit API; not a legacy one sitting beside a modern one, there simply is no 32-bit version. And when IBM did finally build a 32-bit driver environment - KEE, in 1998 - it had to ship KernThunkStackTo16 / KernThunkStackTo32 and a serialization lock, because a 32-bit driver still has to call back down into legacy 16-bit ADDs and that path isn't reentrant. OS/2 started as a 16-bit OS, and the deeper you dig, the more you find. The only real way to ditch 16-bit is to essentially start over, building new DLLs from scratch. My project would have been a lot simpler if I didn't have to handle all of the 16-bit nonsense.

I currently do not intend to support VDM or Win-OS2 - it's just not possible in a traditional sense from a 64-bit kernel. Virtual-8086 mode doesn't exist in long mode, so DOS is out at the CPU level. DOSBox-X / Wine / KVM/QEMU would be a better fit, in my opinion.

Deep in the back burner, my (not completely thought out) idea is this: I've virtualized DOSCALL1.DLL, which is the largest surface between ring-3 OS/2 applications and the kernel. If I created new ordinals for 64-bit versions of the existing calls, one could begin writing new replacements for IBM's DLLs against that interface and maintain compatibility with older applications, while being able to build modern alternatives. However, that's a lot of work, and I haven't looked at it beyond conversations with myself while commuting to work. You need to come up with basically everything, though - do you just make an LX spec that has  ider bits/flags? How or what compiler do you modify to build such a beast? Do you flake out and just use a Windows PE format, or ELF? Do you drop the OS/2 ABI entirely and write directly to the Linux kernel (or something else)? It's a slippery slope.

Your point about 16-bit is totally fair. The reason my project isn't "make a 64-bit OS/2" is because I wanted to provide a transitional medium where we can actually continue to use OS/2 as an operating system.