Improving Keil 8051 Switch Decompilation

[peterbelm]

I'm doing some work on the 8051 support and I'm trying to get a good decompilation of Keils CCASE construct.

For those unaware, Keil will replace some switch statements with a call to CCASE, with a lookup table immediately following the call. This table contains an arbitary length list of 3 byte records with the format:

0	1	2
Handler Addr		Case

The table is terminated with a zeroed handler address, then a default handler address.

My first attempt was to create this call fixup:

val:1 = ACC;
tmp_ptr:2 = inst_next;

<check_case>
addr:2 = *[CODE]:2 tmp_ptr;
tmp_ptr = tmp_ptr + 2;
if (addr == 0) goto <default_case>;

candidate = *[CODE]:1 tmp_ptr;
tmp_ptr = tmp_ptr + 1;
if (val != candidate) goto <check_case>;
goto <go>;

<default_case>
addr = *[CODE]:2 tmp_ptr;
goto <go>;

<go>
PC = zext(addr);
goto [PC];

Which produces output like this (from Keils _PRINTF implementation):

This is certainly better than nothing, it's now at least clear what's happening, but it would be nice to get the switch statement appearing properly.

After looking at the SwitchOverride script I then tried this script to create overrides:

//Override jump tables for Keil 8051 CASE calls
//
// 
//@category Repair

import java.util.ArrayList;
import java.util.List;

import ghidra.app.cmd.function.CreateFunctionCmd;
import ghidra.app.script.GhidraScript;
import ghidra.program.model.address.*;
import ghidra.program.model.data.ByteDataType;
import ghidra.program.model.data.DataUtilities;
import ghidra.program.model.data.DataUtilities.ClearDataMode;
import ghidra.program.model.data.PointerDataType;
import ghidra.program.model.listing.*;
import ghidra.program.model.mem.Memory;
import ghidra.program.model.mem.MemoryAccessException;
import ghidra.program.model.pcode.JumpTable;
import ghidra.program.model.symbol.*;
import ghidra.program.model.util.CodeUnitInsertionException;
import ghidra.util.task.TaskMonitor;
import ghidra.util.exception.CancelledException;
import ghidra.util.exception.InvalidInputException;

public class Keil8051SwitchOverride extends GhidraScript {

    private List<Instruction> collectCalls(Address targetAddr, ReferenceManager refMgr,
            Listing listing, TaskMonitor monitor)
            throws CancelledException {
        List<Instruction> calls = new ArrayList<>();

        for (Reference ref : refMgr.getReferencesTo(targetAddr)) {
            monitor.checkCancelled();

            if (!ref.getReferenceType().isCall())
                continue;

            Address addr = ref.getFromAddress();
            if (addr == null)
                continue;

            Instruction inst = listing.getInstructionAt(addr);
            if (inst == null)
                continue;

            calls.add(inst);
        }

        return calls;
    }

    private void handleCCaseCall(Instruction call, Memory memory, TaskMonitor monitor)
            throws InvalidInputException, CancelledException, MemoryAccessException,
            CodeUnitInsertionException {
        List<Address> destAddrs = new ArrayList<>();
        Address callAddr = call.getAddress();

        Function containingFunc = this.getFunctionContaining(callAddr);
        if (containingFunc == null) {
            println("Call must be in a Function body.");
            return;
        }

        Address ptr = callAddr.add(call.getParsedLength());
        println("ptr = 0x%04X".formatted(ptr.getOffset()));
        short target = 1;

        while (target != 0) {
            monitor.checkCancelled();

            target = memory.getShort(ptr);
            println("ptr = 0x%04X".formatted(ptr.getOffset()));
            println("target = 0x%04X".formatted(target));

            if (target == 0)
                break;

            DataUtilities.createData(currentProgram, ptr, new PointerDataType(), 2,
                ClearDataMode.CLEAR_ALL_CONFLICT_DATA);
            DataUtilities.createData(currentProgram, ptr.add(2), new ByteDataType(), 1,
                ClearDataMode.CLEAR_ALL_CONFLICT_DATA);

            destAddrs.add(ptr.getNewAddress(target));
            ptr = ptr.add(3);
        }

        for (Address address : destAddrs) {
            call.addOperandReference(0, address, RefType.COMPUTED_JUMP, SourceType.USER_DEFINED);
        }

        JumpTable jumpTab = new JumpTable(callAddr, (ArrayList<Address>) destAddrs, true);
        jumpTab.writeOverride(containingFunc);

        CreateFunctionCmd.fixupFunctionBody(currentProgram, containingFunc, monitor);
    }

    @Override
    public void run() throws Exception {
        ReferenceManager refMgr = currentProgram.getReferenceManager();
        Listing listing = currentProgram.getListing();
        Memory memory = currentProgram.getMemory();

        for (Function func : currentProgram.getFunctionManager().getFunctions(true)) {
            println("Function: " + func.getName());
            String fixup = func.getCallFixup();

            if (fixup == null)
                continue;

            println("Fixup: " + fixup);

            switch (fixup) {
                case "ccase":
                    List<Instruction> calls =
                        collectCalls(func.getEntryPoint(), refMgr, listing, monitor);
                    for (Instruction call : calls) {
                        println(
                            "Handling CCASE at 0x%04X".formatted(call.getAddress().getOffset()));
                        handleCCaseCall(call, memory, monitor);
                    }
                    break;
            }
        }
    }
}

And I get this:

Again, better - but now I need to get rid of the loop and have the values as cases rather than the address.

So I figured now would be a good point to take a step back and reach out for some advice. Am I going about this the right way? Is what I'm trying to do possible with the current state of Ghidra? Is this already a solved problem? Where should I go from here?

I'd be grateful for any feedback/advice.

[gtackett]

Thanks for chipping in on this architecture, Peter! I haven't had time to delve in to Keil's generated code very deeply.
As you'll have seen, @ryanmkurtz and I have been working at creating a loader for the OMF-51 object file format.

I've also worked out at least some of the details for a number of record types that Keil added to the OMF-51 object file format. Most of that work, though, is on a private lab network, and is only gradually being integrated into our Ghidra OMF-51 loader.

In my own repo I've also got a branch for the changes to the 8051 language files to better handle the Silicon Labs CIP-51 core architecture. That has been moving a little more slowly than the loader.

Whatever degree of familiarity you have with Keil generated code and/or Keil-specific addtions to OMF-51 may be helpful in both these areas. But we can discuss that elsewhere/else-when.

[peterbelm]

I've implemented most of the record formats I've encountered so far in this PR: #7914

The only one I've not implemented from the files I've looked at are the public debug symbols (0x23), as those are essentially just a duplicate of the public symbols.

I also have a working library filesystem loader ready to go, I'm just not sure if it's good form in this project to add that on to the PR or create a new one - as it is technically a new feature. Any guidance on that would be great!

[ryanmkurtz]

I also have a working library filesystem loader ready to go, I'm just not sure if it's good form in this project to add that on to the PR or create a new one - as it is technically a new feature. Any guidance on that would be great!

@peterbelm A new PR for that would be best. I will try to look at your relocation PR today/tomorrow.

[peterbelm]

@ryanmkurtz thanks for that, I've created a new PR: #7957

[gtackett]

The only one I've not implemented from the files I've looked at are the public debug symbols (0x23), as those are essentially just a duplicate of the public symbols.

I had also worked out the Keil-specific record type 0x23, which I took it to be most similar to Intel's original Debug Items Record type 0x12, You've no doubt seen that In addition to public symbols, record 0x23 can also include segment names, line number symbols, and local symbols, just as Intel's 0x12.

Also, while I've found type 0x23 in the relocatable object files created by the compiler(s) and assembler(s), the very similar type 0x22 is found in absolute object files created by the linker/locator, and in the banked object files that OC51 creates from the latter.

The difference appears to be that where record type 0x23 has a 16-bit field for the segment number in a relocatable object file, record type 0x22 has only an 8-bit field where, if bit 4 is set, bits 0-2 have a code bank number instead.