Utility for comparing binary files

[durgadas311]

This is a project I've been wanting to do for a long time, and finally got the time to work on. I know there are other similar utilities, but the ones I've seen either were not very portable or not useful for my needs. This is written in JAVA and so should easily run (without recompile) anywhere you can install a JRE. Some of the tasks I plan to use it for are checking patched copies of CP/M programs or system components, and comparing (retro system) core dumps (taken at different times/runs) to diagnose problems.

On startup it shows a top-level view of the two files broken up into blocks, with colors indicating the status of each block with respect to being the same or different. Then clicking on a block "zooms in" and shows that block in more detail, eventually showing 1-byte blocks and differences.

I'll start improving it once I have some real-life uses for it (all of my past needs were resolved using other techniques), but it is in a stable enough state to be shared. https://github.com/durgadas311/vdiff.git

 

[Plasma]

Some suggestions: ability to display data as various types (signed/unsigned 8/16/32-bit ints, ASCII text, etc), option to show offsets in decimal, and resynchronizing compare (if one byte is inserted, that would be the only difference, not the entire rest of the file).

 

[durgadas311]

@Plasma Can you expand on usage models for these features? There's already a way to pop-up compare windows in hex and ASCII. Multi-byte integers is a bit different, requiring a choice of big or little endian. I'm wondering what a person is trying to do when they want to see these values.

Being able to "slide" the comparison of the files around would be nice for detecting when there as been a small insertion or deletion from one of the files, but I'm not sure how often that happens. Typically, any such change to code results from recompiling. For ASM it might be more interesting, but for HLLs the changes are probably larger. Still, it's worth spending some time thinking about, to see if there is a way to help determine when a seemingly large different is actually because of a few bytes added/removed. I'll need to come up with an idea of how this would work, and how to implement it without making things much more complicated.

 

[tech58761]

There are a couple existing utilities on macOS with this compare / highlight functionality: BBEdit (text-based files) and Hex Fiend.

 

[durgadas311]

This utility is for comparing two binary files, not editing one. There are good utilities for comparing text files, the one I use in 'meld'. In the past, I've used 'hexdump' to dump the binary files, then 'meld' to compare them. But it has limitations.

 

[Plasma]

@Plasma Can you expand on usage models for these features? There's already a way to pop-up compare windows in hex and ASCII. Multi-byte integers is a bit different, requiring a choice of big or little endian. I'm wondering what a person is trying to do when they want to see these values.

They are all functions that I use frequently when comparing files with Hex Workshop. If two consecutive bytes are different, and they are a 16-bit integer, then it can very useful to see them as the decimal value rather than just two hex bytes.

Being able to "slide" the comparison of the files around would be nice for detecting when there as been a small insertion or deletion from one of the files, but I'm not sure how often that happens. Typically, any such change to code results from recompiling. For ASM it might be more interesting, but for HLLs the changes are probably larger. Still, it's worth spending some time thinking about, to see if there is a way to help determine when a seemingly large different is actually because of a few bytes added/removed. I'll need to come up with an idea of how this would work, and how to implement it without making things much more complicated.

Recompiling is actually a very good example. Because large chunks will be the same, but shifted around. With a simple compare, almost the entire file will show up as different, when it's really not.

 

[durgadas311]

They are all functions that I use frequently when comparing files with Hex Workshop. ...

Hex Workshop looks like an impressive tool, albeit only for Windows. It's a different tool than what I need. But I come from a different environment, where smaller tools that dove-tail with others better fits my work style. Some of those features may make sense, once I can see how a user might employ them in the context of comparing two read-only files. But it's not my intention to create a full-featured code analysis tool, I prefer to keep this machine-agnostic (even ASCII isn't universal to all machines I work with). I can see benefit to a simple sliding of one file in relation to the other, and will consider how that UI would look/work.

 

[tech58761]

This utility is for comparing two binary files, not editing one. There are good utilities for comparing text files, the one I use in 'meld'. In the past, I've used 'hexdump' to dump the binary files, then 'meld' to compare them. But it has limitations.

You were saying?

 

[durgadas311]

Hex Fiend looks interesting, and at least it's open source. But still, only for MacOS. I have nothing against MacOS (and everything against Windows), but I am a Linux shop so look for solutions that are portable or multi-platform. There might be some features to glean from that, if they fit my intended use model. I do not intend this to allow editing of the files, at least for the moment. Patching of CP/M programs is traditionally done in DDT (which makes for more universal distribution), and not as part of a compare operation - although one can certainly see scenarios for that. For disassembly (complete code analysis) I use a modern descendant of DazzleStar (for 8080/Z80 family). So I was looking for a way to quickly assess two copies of the same software to determine what is different. This does not address new versions (releases) of software, only two copies of the "same" software. There have been a lot of times that something is not working with one copy but does work with another, and so having something to show what got corrupted or patched is very useful. In fact I'm dealing with one of those situations right now, and am waiting to get images of the non-working software so I can determine what's different.