Self Modifying Code... Best Practice or avoid?

[Eudimorphodon]

Self modifying on today's processor would be a pain. Code cache and out of order execution would make it hard to do.

It's my vague recollection that a major reason software for the original 68000 Macintosh breaks on 68030 and newer models is because self-modifying code (which was a pretty widespread habit in the early 1980's, especially for the recovering 6502 jockeys that were early developers on the platform) gets broken by the instruction prefetch and caches on the later CPUs, so it's not really a new problem.

I have no idea if there's any Z80 family CPU that it could be a problem with? The eZ80 appears to be "slightly pipelined", but it looks like it might behave differently depending on if it's in Z80-compatible vs. native mode...

 

[Plasmo]

Z280 has cache that is turned on by default, so it needs to be turned off to run self modifying code.
Bill

 

[Chuck(G)]

IN/OUT has already been mentioned wrt 8080 and 8085. No other way to do it with a run-time port specification. Note that SMC is only applicable for processors that can execute from RAM. If you're in ROM-execute only, you're up a stump, unless you want to go to insane lengths. Consider, for example, the code to input a byte from the register given in C on the 8080:

        lxi     d,retloc
        mov     a,c
        shl     a
        shl     a
        mov     c,a
        mvi     b,0
        lxi     h,in_table
        dad     b
        pchl
retloc:
        ...     ; here's your byte

in_table: 
        in      0
        xchg
        pchl

        in      1
        xchg
        pchl
        ...     ; and so on for a total of 256 entries

See? No self-modfying code--just a 1KB table for IN and one for OUT-- and no RAM needed. But horrifically convoluted.

Recall also, that there were processors (e.g. PDP-8) that stored the return address from a JMS in RAM--and that would qualify as self-modifying also.

Some CPUs had an "execute" instruction that could modify the operand of a remote instruction. Description

 

[Makefile]

IN/OUT has already been mentioned wrt 8080 and 8085. No other way to do it with a run-time port specification. Note that SMC is only applicable for processors that can execute from RAM. If you're in ROM-execute only, you're up a stump, unless you want to go to insane lengths. Consider, for example, the code to input a byte from the register given in C on the 8080:

And of course int86() would be like that for the 8086. But I guess you can always construct a stub of code on the stack to avoid generating an inconsistency between in-memory code versus the on-disk binary, if that's what one considers "self-modifying code."

 

[Eudimorphodon]

IN/OUT has already been mentioned wrt 8080 and 8085. No other way to do it with a run-time port specification.

That seems like a really goofy oversight. No wonder the Z80 has so many new flavors of IN/OUT tacked on, it’s overcompensating.

 

[cj7hawk]

And of course int86() would be like that for the 8086. But I guess you can always construct a stub of code on the stack to avoid generating an inconsistency between in-memory code versus the on-disk binary, if that's what one considers "self-modifying code."

I think if the program is written down to memory, and won't ever change between any two executions of the same code, it's not really self modifying though is still self-programming maybe? And I would imagine some of the same problems could still exist as with SMC.

That seems like a really goofy oversight. No wonder the Z80 has so many new flavors of IN/OUT tacked on, it’s overcompensating.

I've noticed people who came from 80/85 think of the z80 as 8 bit I/O while people who came into the z80 only world where it was used ( eg, Spectrum ) consider the z80 to be 16 bit I/O.

I've never been able to find out whether sending B to A8~A15 was intentional or coincidental.

But it sure is nice to have

 

[Eudimorphodon]

I've noticed people who came from 80/85 think of the z80 as 8 bit I/O while people who came into the z80 only world where it was used ( eg, Spectrum ) consider the z80 to be 16 bit I/O.

The manual says it has 256 ports. That’s what it has.

Anyway, for the scope of this discussion I was referring to how all the alternate flavors use the contents of register C as the port number instead of encoding it the way the 8080 does; that is potentially a big deal for a ROM based system with scant enough RAM to make rigging up a Mickey Mouse workaround painful. (Granted it seems like you’d only need a few bytes.)

 

[Chuck(G)]

The manual says it has 256 ports. That’s what it has.

Well, maybe. There is a bit of a hack where an indirect single byte {e.g. in a,(c) ] places the contents of the B register on the upper 8 bits of the address bus. So there, the limit is 65K I/O addresses. I think it was actually used in practice. This works only for single-byte I/O indirect port instructions, as the multi-byte ones use B as a repeat counter.

 

[WSM]

Just to further confuse the Z80 and it's derivatives 16-bit I/O characteristics: While IN r,(C) places the B register on A8-A15, IN A, places the A register onto A8-15.

 

[Chuck(G)]

Isn't that well-documented? I have no idea of what goes on A8-A15 in the case of the undocumented ED 70 and ED 71 I/O instructions--or even if it's useful.

 

[Eudimorphodon]

Well, maybe. There is a bit of a hack...

Yeah, I'm aware; there was a recent thread where we all talked until we were blue in the face about it, which I was referencing/lampshading with that comment. but if we're going to argue semantics again...

A"plain reading" of the current version of the Z80 user manual published by Zilog makes it clear that *officially* the "port address" is considered to be the bottom 8 bits of the address bus. They do document what bleeds out to the upper half, and that they have kept that behaviour in subsequent versions of the architecture can certainly be taken as unofficially acknowledging/endorsing making use of those bits however people choose to use them, but again, for instance, the description for IN r (C) reads:

The contents of Register C are placed on the bottom half (A0 through A7) of the address bus to select the I/O device at one of 256 possible ports...

This exact sentence is repeated in the description for every one of the Z80's alternate I/O instructions. So, sure, call the extra goodies on the other half of the address bus a 16 bit address, an 8 bit data value (an alternate way this has been used by people; for instance, it's a clever way to program a memory mapper like the 74LS612 without external latches), a "function selector", whatever, Zilog doesn't care what you do behind closed doors, but officially it has an 8 bit I/O address range just like the 8080.

Anyway, back on topic, it's worth noting that the Z80 manual on page 33 specifically calls out that an advantage of their new versions of the I/O instructions is they make it a lot easier to reuse driver code when it's stored in ROM. I would guess more than one Intel customer complained about this.

The input/output group of instructions in the Z80 CPU allow for a wide range of transfers between external memory locations or the general-purpose CPU registers, and the external I/O devices. In each case, the port number is provided on the lower eight bits of the address bus during any I/O transaction. One instruction allows this port number to be specified by the second byte of the instruction while other Z80 instructions allow it to be specified as the contents of the C Register. One major advantage of using the C register as a pointer to the I/O device is that it allows multiple I/O ports to share common software driver routines. This advantage is not possible when the address is part of the op code if the routines are stored in ROM.

 

[Chuck(G)]

I recall the discussion. My own view is if the behavior is officially documented, then use it for whatever floats your boat. After reading lots of MCU manuals, I'm not going to cry about documentation quirks. There's enough in those bundles of verbiage to cause a nosebleed.

Consider the "undocumented" 8085 instructions. Coding orthodoxy would demand that they not be used. On the other hand, they were later documented by a licensee (Tundra in their 80C85 docs). Every second-source 8085 that I've been able to check has them. As someone who has written a ton of 8085 code, I was a bit miffed by the late mention, as we could have used a few of those instructions to good advantage. Taking the official Intel point of view, I can see that use would make "perfect" automated conversion to 8086 code more difficult.

On the other hand, there are tons of officially undocumented Z80 codes--and I wouldn't be caught dead using any of them.

 

[cj7hawk]

Isn't that well-documented? I have no idea of what goes on A8-A15 in the case of the undocumented ED 70 and ED 71 I/O instructions--or even if it's useful.

A reasonable expectation would be that B is placed on the upper address lines for all of those commands. It's use would depend on the hardware for a particular machine architecture.

In my case, with plans to support an ISA bus with z80, it would allow PC hardware to be used by extending A8 and A9 onto the bus, but would be potentially incompatible with software that used 8 bit addresses. Though it's not so much of a problem as if we take 3F8 ( Serial ) it would spread the serial ports like 0F8, 1F9, 2FA, 3FB etc.... Only 2FA and 3FB would typically be valid, and even then if nothing existed at the other addresses, it's not an issue. So it's a reasonable approach so long as all ports are mapped only in 8 bit I/O space and PC cards are not anticipated to be used by older software.

Though specific use of those two command are uncertain, because like any undocumented command, it may or may not exist, but inputting to C should be possible, as is writing a 0 to the 16-bit address "BC" - I could probably use the second to clear the DMA buffer if there was any value in doing so, and the first to read a port specified by an earlier port, eg,

IN C,(C)
IN A,(C) - Would allow the port to be read from to be specified by an external source...

Well, it's contrived usefulness and It's a cool feature but I'm not sure where I'd use it. On the contrary, I use INDR quite a bit as a 16 bit I/O space address for moving CP/M's DMA space between RAMDISK/DISK and Main Memory as my hardware shifts the upper A8 to A14 down to A0 to A6, and I place disk track and sector on the upper addresses. It only works for INDR and OTDR since B is decremented, as is DE in that instance, allowing them to stay in lockstep and that command *is* documented.

But yes, like Eudimophodon, I too was referencing the earlier thread -

 

[Eudimorphodon]

I recall the discussion. My own view is if the behavior is officially documented, then use it for whatever floats your boat.

I actually brought up the memory mapper programming example because I have some vague (semi-hashed out in Kicad) plans to use it like that, so, yeah, not saying you shouldn’t use it for whatever. I just don’t think it’s accurate to call it “16 bit addressing” for the very reason that Zilog *clearly* doesn’t prescribe or define that use for it.

(If we really wanted to get into the weeds about it I would also question why in the world, if Zilog really intended to expand the Z80 to “16 bit addressing“ officially, would they implement the auto-repeat versions of the port addressing commands in this broke-a$$ way with the 8 bit counter and 8 bit port address sharing the BC register pair instead of just making the extended port I/O commands essentially aliases of LDD/LDIR, IE, with the counter and address stored separately.)

And FWIW, the way the Sinclair uses it isn’t really 16 bit addressing either; the Sinclair essentially is designed to use the 16 address lines as direct chip selects, most combinations with more than one bit set are invalid. It’s a clever hack for maximizing cheapness, but again, it’s definitely not ”16 bit I/O addressing” in the sense of, say, an 8088.

 

[Chuck(G)]

No, for that you'd use a Z8000. My hellbox has a bunch of Z8002 PLCCs (basically non-segmented/64KB address space Z8000). A "never got to it" project was to do something with them. Same for my NS32016s. But they'd both be dead-end projects.

 

[Svenska]

But, to your point, in this day and age one could say there is a distinction between SMC and JIT.

Yes, and "in this day and age" SMC is basically banned on security grounds. Code is marked non-writable and executable, data and stack are writable and non-executable, and this is enforced by hardware on page boundary levels. JIT is done by asking the operating system to bypass these mechanisms.

Unless you are on iOS, where any form of JIT is simply banned by policy.

I think if the program is written down to memory, and won't ever change between any two executions of the same code, it's not really self modifying though is still self-programming maybe?

Copying unmodified code from ROM to RAM is commonly done on microcontrollers. Executing code from embedded flash forces wait-states, so performance-critical code is executed from RAM. I'd see this as a form of manual caching, so a performance optimization.

It is also common to build dynamic data structures in RAM (for example, processing pipelines) ahead of time to execute it faster. In a data-driven code base, this isn't really different from JITs (obviously, it is different because instruction bytes stay unchanged). Many FORTH implementations fall under this category, as they chain existing code blocks rather than change them.

I've never been able to find out whether sending B to A8~A15 was intentional or coincidental.

How would you differ between the two cases?

It's a side-effect of the hardware implementation; it was known to Zilog; it is useful; not doing it this way would have required more work + increased complexity + raised the price.

 

[Phil_G]

My Logiker Christmas 2023 entry (sc/mp assembler) did one byte of smc

 

[Plasmo]

The 16-bit I/O space is useful to me in two designs:
1. 4K video memory and fonts are mapped to 4K of I/O so they are directly read/write accessible.
2. Multiprocessor communication over dual port RAM where main processor access its side of 4K dual port RAM as 4K of I/O, thus allowing up to 16 dual-port RAM based slave processors.

Z280 not just retained the 16-bit I/O capability, they extended it to 24 bits.
Bill

 

[WSM]

Zilog's eZ80 processors (except eZ80190) force all I/O addresses to be 16-bit, regardless of being in Z80 or ADL mode. Per the product specifications: "All I/O operations employ 16-bit addresses" and "... addresses within the 0000h-00FFh range are routed to the on-chip peripherals". Also, "The upper byte of the 24-bit address bus is undefined during all I/O operations ...".

 

[resman]

Yes, and "in this day and age" SMC is basically banned on security grounds. Code is marked non-writable and executable, data and stack are writable and non-executable, and this is enforced by hardware on page boundary levels. JIT is done by asking the operating system to bypass these mechanisms.

Unless you are on iOS, where any form of JIT is simply banned by policy.

... on protected mode operating systems. I suppose you could find a way around it, just as JIT compilers do, but I can't even imagine a scenario where this makes the least bit of sense.

Modern embedded processors (at least the ones I've used) have learned from the errors of their 8 bit forefathers where such shenanigans are no longer needed.