What kind of binary executable file format did the Apple Lisa use?

[hippietrail]

I'm assuming it used something proprietary and unique and not a.out or ELF.
Was it anything like the `APPL`/`CODE` resource format used on the early Macs? Maybe something more like the "bundles" we see today?

Or was it something totally exotic?

(Typical Googling for related terms turns up nothing remotely technical or interesting.)

 

[commodorejohn]

Curious about this myself, but I doubt there was much similarity to MacOS on the backend; aside from the shared PARC heritage they were very much rival projects within Apple, and I don't think there was a lot of sharing of notes going on.

 

[whartung]

I don't know.

It would not surprise me at all if it was very similar to what the Mac had. The Macs Segment based executable format is very similar to what UCSD did, and with the Pascal heritage, as well as a lot of UCSD heritage at early Apple, it would not surprise me at all if they used that in the Lisa system as well. Theres a lot of Lisa in the Mac.

The Segment based format was also used in the Apple II GS, and it's a quite powerful format allowing for easy overlays and dynamic loading, and swapping, of code even on a rather primitive system.

 

[stepleton]

This should be possible to figure out based on the Lisa OS source code, but I am not sure where to look.

I don't know about the Segment system at all, but I had some thoughts.

It happens that the OS supported shared libraries or "intrinsic libraries" as they were called. If the Segment system permitted this too, then the hypothesis survives, but if it didn't, then an alternative is needed.

The Lisa's MMU would allow the OS to remap the address space available to applications in a way that the early Mac and IIGS hardware would not be capable of doing, so far as I'm aware. It would be strange for the OS not to take advantage of this capability, and I think it's probably fairly safe to assume that it does. Therefore if Segment does not allow for this, then that too could shed doubt on the hypothesis.

 

[resman]

I do have the Lisa Workshop 3.0 but it doesn't explicitly detail the file format. However, as whartung said, it was heavily influenced if not directly derived from the UCSD Pascal system so its codefile format is probably very similar. There is documentation on how to set up segments to be on-demand loaded, although not sure exactly how that fits in with the MMU.

 

[Bob-O-Rama]

I would start with getting...

Download Apple Lisa Source Code

info.computerhistory.org

... and unzipping it and looking for salacious key words. ( grep -R is your friend ) I mean only some many of them will relate to loading binary executibles into memory. Also a linker would need to be able to package them as well. Something that provied properties for a binary file - like Finder, etc may also have some of that. Once you have a thread to pull, yank and it should lead to an underlying source and header.

 

[hippietrail]

I would start with getting...

Download Apple Lisa Source Code

info.computerhistory.org

... and unzipping it and looking for salacious key words. ( grep -R is your friend ) I mean only some many of them will relate to loading binary executibles into memory. Also a linker would need to be able to package them as well. Something that provied properties for a binary file - like Finder, etc may also have some of that. Once you have a thread to pull, yank and it should lead to an underlying source and header.

It turns out somebody has created a code navigator especially for the Lisa source code. Apparently the source for the Pascal compiler is in there!

Apparently it's archived as DiskCopy 4.2 format disk images, a format CiderPress doesn't support but apparently AppleSauce does.

I haven't delved into the source yet but I've looked through some Lisa programming manuals and it does talk about segments in relation to processes, but I didn't find anything about loading executable files. Lots of OSes had segmented executable files of course, in a wide variety of formats. I had forgotten that IIGS also used a very similar resource file format to classic Mac so that's a strong hint the Lisa format could be similar.

I'm actually working on a project to load many retro binary executable file formats and classic Mac was one of the first I got working. It didn't really use segments but some resources came to be used pretty much like segments I believe.

If anyone knows where I can get any small Lisa executable files from I would appreciate it.

 

[commodorejohn]

I haven't delved into the source yet but I've looked through some Lisa programming manuals and it does talk about segments in relation to processes, but I didn't find anything about loading executable files. Lots of OSes had segmented executable files of course, in a wide variety of formats. I had forgotten that IIGS also used a very similar resource file format to classic Mac so that's a strong hint the Lisa format could be similar.

Well, GS/OS specifically draws heavily on classic MacOS, and in fact uses the same filesystem (HFS,) which includes the capability for files to have a "resource fork" in addition to a "data fork;" IIRC executable segments are treated as just a type of resource. (folklore.org indicates that the division of programs into segments was done to enable position-independent loading of code into limited memory space on-demand; the 32KB limit on segment size comes from the boundary of a relative branch/jump on the 68k.) I'm unclear about the Lisa OS and whether it was even a forked filesystem; a brief Wikipedia skim indicates that the Lisa filesystem was based on the SOS/ProDOS filesystem (which wasn't, AFAIK,) but I dunno what they did or didn't add to it.

 

[hippietrail]

I found another Lisa Manual that I missed. Lisa Development Systems Internals. The "Lisa Object File Formats" section starts at PDF page 102.

So far I've seen nothing hinting at a forked filesystem. The object file format seems similar to what I've seen described as a "tagged object file" on other systems where it's a series of 1-byte type identifiers followed by a 3-byte length field followed by 'length' bytes of data. There doesn't seem to be an overall header field with any kind of signature or magic word. There's also no four-character-codes.

Number 98, "Main Program" seems to be built upon a jump table, which reminds me of Classic Mac program structure as CODE resource 0 etc. The first program I poked around in, "system.shell", has one that's numbered A0, which is not mentioned in the PDF so I wonder what that one is.

 

[whartung]

Number 98, "Main Program" seems to be built upon a jump table, which reminds me of Classic Mac program structure as CODE resource 0 etc.

Yea, that the gist of the segment system. MacOS, GSOS, and UCSD worked similarly (I just assume Lisa did also).

In order to invoke a procedure, you didn't just jump to an address, rather you setup the stack, and made a system call effectively saying "I want to call procedure #123 in Segment #456", and the system did all of the work from there. Notably, locating the segment, loading the segment, even evacuating no longer used segments. The segment loader is very tightly tied to the memory manager. The structure of a segment had the locations of the code with some kind of TOC at the top of it (the jump table).

Anyone who ran an old 128K Macintosh and was playing floppy swap hell has experienced this system first hand, all that floppy swapping is a memory starved system loading up segments. Since it "knows" what disk the program is on, it knows which one to ask for when it wants to load some more code. Proper code would load a segment, lock it, use it, unlock it, and move on. With low memory, those segments were loaded, used, discarded, and something else was reloaded.

Very impactful performance wise, but a nice way to empower developers to make large programs in the constrained memory environments of the day.

When I learned UCSD had something that sophisticated, for such an early system, I was pretty amazed. In hindsight, it all makes sense and is a pretty elegant system.

 

[Bob-O-Rama]

Overlays have been done since 1957, and on microcomputers since 1972. Swap devices made all of that shenanigans entirely unnecessary and developers did not need to be burdened by the internals of the operating system or the hardware. The way that the 128K gave way to the 512K showed the limits of that approach were reached rather rapidly.

Anyway, just looking through the sources, I think some of the requested items are in the Toolkit sources. There seemed to be a section related to loading / swapping segments - also building binaries. its a bit obscure because instead of "loading stuff from files" which apple is way too cool for, they enshroud it in a lot of oddball terms that they seems to have just made up. Also there is a Lisa emulator project out there, which means likely Lisa binaries are also available. So you may be able to directly obtain them and do some hexdumping.

 

[hippietrail]

Overlays have been done since 1957, and on microcomputers since 1972. Swap devices made all of that shenanigans entirely unnecessary and developers did not need to be burdened by the internals of the operating system or the hardware. The way that the 128K gave way to the 512K showed the limits of that approach were reached rather rapidly.

Yes Amiga had overlays too though they were poorly documented and barely ever used. Several of the other OSes whose binary executable file formats I've been exploring had them too. Amiga also had similar early RAM story, though it originally had 256K that quickly gave way to 512K. We also had lots of floppy disk swapping.

Anyway, just looking through the sources, I think some of the requested items are in the Toolkit sources. There seemed to be a section related to loading / swapping segments - also building binaries. its a bit obscure because instead of "loading stuff from files" which apple is way too cool for, they enshroud it in a lot of oddball terms that they seems to have just made up. Also there is a Lisa emulator project out there, which means likely Lisa binaries are also available. So you may be able to directly obtain them and do some hexdumping.

Haha yes writing a loader for Classic Mac binaries was trickier than most, but also fun.
I haven't downloaded the Lisa sources yet but I did find disk images of the OS and unpacked them with AppleSauce and quickly wrote the code that can identify which files are object files. I also downloaded LisaEm and noted updates abruptly stopped in 2022 I think then confirmed that sadly its developer passed away.
I haven't tried to set up ROMs and install an OS on it yet. AppleSauce has quite a nice hex viewer of disk images, sectors, files, etc, which helped me see some files were in the object format.
I should be able to get code disassembling properly with just a bit more perusal of the docs now that I have some files.
I'm not yet sure about the differences between pre-link object files, library files, and executable files, but I expect it will be much like other systems.

 

[stepleton]

I'm unclear about the Lisa OS and whether it was even a forked filesystem; a brief Wikipedia skim indicates that the Lisa filesystem was based on the SOS/ProDOS filesystem (which wasn't, AFAIK,) but I dunno what they did or didn't add to it.

I think there are at least two Lisa OS filesystems and maybe 3, roughly corresponding to 1.0, 2.0, and 3.0 of the Office System/Workshop. 1.0 is a flat filesystem and 3.0 is hierarchical (see the Workshop 3.0 supplement). I suspect the Wikipedia article is incorrect or at least skims over a lot of detail.

I'm not aware that files have anything like the resource fork per se, but there is room for 128 bytes of user- or program-supplied metadata called the "label".

The filesystem also supports other oddities like named pipes. They were deprecated by 3.0 and supposed to be removed in a later release IIRC, but that day never came. (Maybe it did with 3.0.)

A long, long time ago I wrote a program for v1.0 of the Workshop that would allow you to download files to a modern computer over a serial cable. (Buggy, but it worked for individual files.) At the time this seemed like a potential workaround for not having tools on a modern computer that understand the filesystem. Eventually someone reverse-engineered the filesystems and put the result into this tool, but that tool was not something I ever tried myself. Since then more information about the filesystem has been revealed thanks to the Office System source code release, of course.

If anyone knows where I can get any small Lisa executable files from I would appreciate it.

The links above may offer some options: if you have a Lisa running the Workshop, you can try to build my tool on it (may need updates for 3.0) and then download some executable files. Or, you could install a Lisa OS environment (i.e. Workshop or Office System) in LisaEm and then use the other linked tool to extract the files you want. Your best bet is probably the small utilities bundled with the Workshop; or, since you have the Workshop, just build and compile a Hello World program. You could also apply the second tool to this disk image, where you'll find a Mandelbrot set viewer program that I wrote for the Workshop. (Or any of the Workshop installation disks would work too.)

Never forget: TST.W before you leap!

 

[hippietrail]

You could also apply the second tool to this disk image, where you'll find a Mandelbrot set viewer program that I wrote for the Workshop. (Or any of the Workshop installation disks would work too.)

Oh this is actually perfect. Thank you. I've managed to get it disassembling in Ghidra well enough that I can already see that LisaOS also used the 68000's A-line traps to do syscalls.

 

[Al Kossow]

I'm saddened that so little has been done with the source code I worked to release
and gathering up all of the documentation and software on bitsavers.

The fact that the Lisa was developed using the Silicon Valley Software 68000 Pascal
compiler should give you a loaf sized breadcrumb where to look for the runtime
information along with the internal documents on bitsavers.

No knowledge of any of other Apple system is needed or required and in most
cases it's going to only confuse you.

Another huge hint: Resource forks were invented for the Mac and are completely
irrelevant to Lisa.

 

[Eschaton]

Unfortunately the missing pieces (the missing EXEC files that were caused by the filesystem-extraction tool having bugs, not the couple of components that couldn’t be redistributed), and the poor state of Lisa emulation make it very difficult to build or do much with besides browse. And then if, say, bugs were fixed or enhancements added, the license may be too restrictive to allow them to be shared unless one is sufficiently brave.

 

[Eschaton]

I’ll look through the code and post back here with some pointers to where the runtime loader can be found in it.

Remember that Lisa VM system is itself cooperative: It relies on the Pascal compiler inserting “probes” that can pre-trigger VM fault interrupts that are safely resumable, rather than relying on bus errors (which aren’t resumable on 68000, something that was fixed for 68010).

 

[stepleton]

I'm saddened that so little has been done with the source code I worked to release
and gathering up all of the documentation and software on bitsavers.

I'm still convinced that this is just a matter of time. If I'm ever temporarily marooned on an internet-free island with my personal laptop (and therefore far away from that janky old PERQ), then I'd probably make the Lisa source code my main preoccupation.

I fear this could be an annoying question, but do you suppose it might ever be possible to be able to link to parts of the source code from a web browser? I think this could make it easier to have conversations online about understanding the source code --- for example, in the discussion above, we could be trading links to different parts of the OS that we thought might be responsible for the executable format. I'd hope discussions like these could spark a lot of interest and excitement about exploring the system.

I know about the source code browser, but I think a native desktop program isn't quite as handy as visiting a webpage --- being able to scroll it on a phone or on any computer in idle moments would multiply the opportunities to poke around (at least for me).

 

[Eschaton]

OK, I've looked through the Lisa source license again to pinpoint the problem with it.

The good news: It doesn't restrict distributing modifications of the code.

The bad news: It restricts distributing the code itself (referred to in the license as "publication")

This is an utterly ridiculous non-starter in the modern era. It means nobody is technically allowed to put a repository up on GitHub that people can use for collaboration—in fact, it even means that technically someone can't distribute patches that contain substantial amounts of surrounding context! To comply with the letter of the license, one would have to distribute "blind" patches (e.g. "replace line number X of file Y with text Z") or be prepared to defend the patch context as being too inconsequential as to be covered by copyright.

It's a really stupid license that Apple and CHM have put on this. They should really do the work to revise the license to not prevent publication. Even if it only granted an exception for forking from an official github.com/apple/lisa repository that would be sufficient to let people actually do something with the code beyond stare at it.

 

[NicholasLee]

Lisa used a format called "Lisa Office System Object", or "LOSO" for short. It's described in some detail in the Lisa Workshop manual.

Not really anything like the later Mac stuff, though. (And also not really all that exotic, just different.)