4 posts tagged with "Solidity"

Transparency, verifiability, and trustlessness are the core values of blockchains and Ethereum especially. We want the smart contracts we are interacting with to be open-source (right?). However you can't be sure if the open-source code you see is actually the one that lives on chain. I can show you a benign code on GitHub etc. and convince you to send your assets to a contract, but in reality it could be a malicious contract that's actually deployed at this address.

This is where source code verification comes in. Source code verification makes sure the human-readable source code you see is the same as the one that was deployed on chain.

Smart contracts are written in human-readable programming languages like Vyper or Solidity. But they are compiled to and deployed in bytecode (1s and 0s), so they are not human-readable.

0x608060405234801561001057600080fd5b50600436106100365760003560e01c80632e64cec11461003b5780636057361d14610059575b600080fd5b610043610075565b60405161005091906100d9565b60405180910390f35b610073600480360381019061006e919061009d565b61007e565b005b60008054905090565b8060008190555050565b60008135905061009781610103565b92915050565b6000602082840312156100b3576100b26100fe565b5b60006100c184828501610088565b91505092915050565b6100d3816100f4565b82525050565b60006020820190506100ee60008301846100ca565b92915050565b6000819050919050565b600080fd5b61010c816100f4565b811461011757600080fd5b5056fea2646970667358221220404e37f487a89a932dca5e77faaf6ca2de3b991f93d230604b1b8daaef64766264736f6c63430008070033

We can go from Solidity/Vyper code to bytecode, but not the other way around. This information is lost during compilation and we need the original source-code that compiles down to this bytecode.

In simple terms, source code verification works by:

1. Taking a smart-contract written in a human-readable programming language (Solidity/Vyper)
2. Compiling it down to bytecode
3. Comparing the compiled bytecode with the on-chain bytecode that is deployed at a certain chain and address.

Of course this is a simplified explanation and there are some nuances to it. In this blog post, we will dive deeper into the technical details of the process and walkthrough how a verification works behind the scenes.

How to Verify?​

When you want to verify a contract on Sourcify we need the following:

• chainId where the contract is deployed
• address of the contract
• metadata.json: The Solidity Contract Metadata file. This JSON file is output by the compiler and contains information about how to interact with this contract (abi, userdoc, devdoc), and how to reproduce the compilation (compilation settings, source hashes or contents)
• Source files outlined in the metadata.json.

Say we wanted to verify the contract 0x48A331150C1b442444F4f0371a4daC9Ab2FC837D on Holesky Testnet.

User request to Sourcify:

{
  "address": "0x48A331150C1b442444F4f0371a4daC9Ab2FC837D",
  "chainId": "17000", // Ethereum Holesky testnet
  "files": {
    "metadata.json": "{\"compiler\":{\"version\":\"0.8.7+commit.e28d00a7\"},\"language\":\"Solidity\",\"output\":{\"abi\":[{\"inputs\":[],\"name\":\"retrieve\",\"outputs\":[{\"internalType\":\"uint256\",\"name\":\"\",\"type\":\"uint256\"}],\"stateMutability\":\"view\",\"type\":\"function\"},{\"inputs\":[{\"internalType\":\"uint256\",\"name\":\"num\",\"type\":\"uint256\"}],\"name\":\"store\",\"outputs\":[],\"stateMutability\":\"nonpayable\",\"type\":\"function\"}],\"devdoc\":{\"details\":\"Store & retrieve value in a variable\",\"kind\":\"dev\",\"methods\":{\"retrieve()\":{\"details\":\"Return value \",\"returns\":{\"_0\":\"value of 'number'\"}},\"store(uint256)\":{\"details\":\"Store value in variable\",\"params\":{\"num\":\"value to store\"}}},\"title\":\"Storage\",\"version\":1},\"userdoc\":{\"kind\":\"user\",\"methods\":{},\"version\":1}},\"settings\":{\"compilationTarget\":{\"contracts/1_Storage.sol\":\"Storage\"},\"evmVersion\":\"london\",\"libraries\":{},\"metadata\":{\"bytecodeHash\":\"ipfs\"},\"optimizer\":{\"enabled\":false,\"runs\":200},\"remappings\":[]},\"sources\":{\"contracts/1_Storage.sol\":{\"keccak256\":\"0xb6ee9d528b336942dd70d3b41e2811be10a473776352009fd73f85604f5ed206\",\"license\":\"GPL-3.0\",\"urls\":[\"bzz-raw://fe52c6e3c04ba5d83ede6cc1a43c45fa43caa435b207f64707afb17d3af1bcf1\",\"dweb:/ipfs/QmawU3NM1WNWkBauRudYCiFvuFE1tTLHB98akyBvb9UWwA\"]}},\"version\":1}",
    "1_Storage.sol": "// SPDX-License-Identifier: GPL-3.0\n\npragma solidity >=0.7.0 <0.9.0;\n\n/**\n * @title Storage\n * @dev Store & retrieve value in a variable\n */\ncontract Storage {\n\n    uint256 number;\n\n    /**\n     * @dev Store value in variable\n     * @param num value to store\n     */\n    function store(uint256 num) public {\n        number = num;\n    }\n\n    /**\n     * @dev Return value \n     * @return value of 'number'\n     */\n    function retrieve() public view returns (uint256){\n        return number;\n    }\n}"
  },

For now the metadata.json is the main way to submit contracts to Sourcify. All other methods such as "Import from Etherscan" all workaround to generate the metadata file in some way, and continue verification from there. This is will change with the Vyper verification and in our APIv2. In Vyper, users will give us a standard JSON and we generate a "fake" metadata.json for backward compatability reasons. In our APIv2 we will no longer use the metadata file as the base of our verification, but the standard JSON as the base.

{
  "compiler": { "version": "0.8.7+commit.e28d00a7" },
  "language": "Solidity",
  "output": {
    "abi": [
      {
        "inputs": [],
        "name": "retrieve",
        "outputs": [{ "internalType": "uint256", "name": "", "type": "uint256" }],
        "stateMutability": "view",
        "type": "function"
      },
      {
        "inputs": [{ "internalType": "uint256", "name": "num", "type": "uint256" }],
        "name": "store",
        "outputs": [],
        "stateMutability": "nonpayable",
        "type": "function"
      }
    ],
    "devdoc": {
      "details": "Store & retrieve value in a variable",
      "kind": "dev",
      "methods": {
        "retrieve()": { "details": "Return value ", "returns": { "_0": "value of 'number'" } },
        "store(uint256)": { "details": "Store value in variable", "params": { "num": "value to store" } }
      },
      "title": "Storage",
      "version": 1
    },
    "userdoc": { "kind": "user", "methods": {}, "version": 1 }
  },
  "settings": {
    "compilationTarget": { "contracts/1_Storage.sol": "Storage" },
    "evmVersion": "london",
    "libraries": {},
    "metadata": { "bytecodeHash": "ipfs" },
    "optimizer": { "enabled": false, "runs": 200 },
    "remappings": []
  },
  "sources": {
    "contracts/1_Storage.sol": {
      "keccak256": "0xb6ee9d528b336942dd70d3b41e2811be10a473776352009fd73f85604f5ed206",
      "license": "GPL-3.0",
      "urls": [
        "bzz-raw://fe52c6e3c04ba5d83ede6cc1a43c45fa43caa435b207f64707afb17d3af1bcf1",
        "dweb:/ipfs/QmawU3NM1WNWkBauRudYCiFvuFE1tTLHB98akyBvb9UWwA"
      ]
    }
  },
  "version": 1
}

Anyway for now we continue with our metadata.json. To be able to compile we first need to make sure we have everything we need, that are, all the source files and settings outlined in the metadata.json. The JSON file has a sources field that looks like this

  "sources": {
    "contracts/1_Storage.sol": {
      "keccak256": "0xb6ee9d528b336942dd70d3b41e2811be10a473776352009fd73f85604f5ed206",
      "license": "GPL-3.0",
      "urls": [
        "bzz-raw://fe52c6e3c04ba5d83ede6cc1a43c45fa43caa435b207f64707afb17d3af1bcf1",
        "dweb:/ipfs/QmawU3NM1WNWkBauRudYCiFvuFE1tTLHB98akyBvb9UWwA"
      ]
    },
    ...
  }

And with the file hashes we can validate the source files user gave us and make sure we have all the source files needed. Taking the user request, we:

1. Find the metadata file in the user request .files (assume the rest are source files)
2. Hash all other files and keep their keccak
3. Additionally generate new line and whitespace variations of the source files to account for the OS and platform differences.
4. Try to mark out all entries in sources according to their keccak256
5. If there are sources missing, try to fetch them from their IPFS hash e.g. dweb:/ipfs/Qmf2J3oWXHBnoNcGXkNYcSirUvJZebXa3j3Cn3vccqS1x7
6. If we still have missing sources, tell user what's missing

Assuming we found all the sources we go forward with the compilation with the language, compiler.version and the settings field in the metadata.json:

{
  "compiler": {
    "version": "0.8.7+commit.e28d00a7"
  },
  "language": "Solidity",
  ...
  ...
  "settings": {
    "compilationTarget": {
      "contracts/1_Storage.sol": "Storage"
    },
    "evmVersion": "london",
    "metadata": {
      "bytecodeHash": "ipfs"
    },
    "optimizer": {
      "enabled": false,
      "runs": 200
    },
  },

With this information we create a standard JSON input file and feed it to the compiler.

The compiler will give us an output that looks like this:

{
  "contracts": {
    "contracts/1_Storage.sol": {
      "Storage": {
        "abi": [{"inputs":[],"name":"retrieve","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"uint256","name":"num","type":"uint256"}],"name":"store","outputs":[],"stateMutability":"nonpayable","type":"function"}],
        "devdoc": {"details":"Store & retrieve value in a variable","kind":"dev","methods":{"retrieve()":{"details":"Return value ","returns":{"_0":"value of 'number'"}},"store(uint256)":{"details":"Store value in variable","params":{"num":"value to store"}}},
        "evm": {
          "bytecode": {
            "generatedSources": [],
            "linkReferences": {},
            "object": "608060405234801561001057600080fd5b50610150806100206000396000f3fe608060405234801561001057600080fd5b50600436106100365760003560e01c80632e64cec11461003b5780636057361d14610059575b600080fd5b610043610075565b60405161005091906100d9565b60405180910390f35b610073600480360381019061006e919061009d565b61007e565b005b60008054905090565b8060008190555050565b60008135905061009781610103565b92915050565b6000602082840312156100b3576100b26100fe565b5b60006100c184828501610088565b91505092915050565b6100d3816100f4565b82525050565b60006020820190506100ee60008301846100ca565b92915050565b6000819050919050565b600080fd5b61010c816100f4565b811461011757600080fd5b5056fea2646970667358221220404e37f487a89a932dca5e77faaf6ca2de3b991f93d230604b1b8daaef64766264736f6c63430008070033",
            "sourceMap": "141:356:0:-:0;;;;;;;;;;;;;;;;;;;"
          },
          "deployedBytecode": {
            "immutableReferences": {},
            "linkReferences": {},
            "object": "608060405234801561001057600080fd5b50600436106100365760003560e01c80632e64cec11461003b5780636057361d14610059575b600080fd5b610043610075565b60405161005091906100d9565b60405180910390f35b610073600480360381019061006e919061009d565b61007e565b005b60008054905090565b8060008190555050565b60008135905061009781610103565b92915050565b6000602082840312156100b3576100b26100fe565b5b60006100c184828501610088565b91505092915050565b6100d3816100f4565b82525050565b60006020820190506100ee60008301846100ca565b92915050565b6000819050919050565b600080fd5b61010c816100f4565b811461011757600080fd5b5056fea2646970667358221220404e37f487a89a932dca5e77faaf6ca2de3b991f93d230604b1b8daaef64766264736f6c63430008070033",
            "sourceMap": "141:356:0:-:0;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;416:79;;;:::i;:::-;;;;;;;:::i;:::-;;;;;;;;271:64;;;;;;;;;;;;;:::i;:::-;;:::i;:::-;;416:79;457:7;482:6;;475:13;;416:79;:::o;271:64::-;325:3;316:6;:12;;;;271:64;:::o;7:139:1:-;53:5;91:6;78:20;69:29;;107:33;134:5;107:33;:::i;:::-;7:139;;;;:::o;152:329::-;211:6;260:2;248:9;239:7;235:23;231:32;228:119;;;266:79;;:::i;:::-;228:119;386:1;411:53;456:7;447:6;436:9;432:22;411:53;:::i;:::-;401:63;;357:117;152:329;;;;:::o;487:118::-;574:24;592:5;574:24;:::i;:::-;569:3;562:37;487:118;;:::o;611:222::-;704:4;742:2;731:9;727:18;719:26;;755:71;823:1;812:9;808:17;799:6;755:71;:::i;:::-;611:222;;;;:::o;920:77::-;957:7;986:5;975:16;;920:77;;;:::o;1126:117::-;1235:1;1232;1225:12;1249:122;1322:24;1340:5;1322:24;:::i;:::-;1315:5;1312:35;1302:63;;1361:1;1358;1351:12;1302:63;1249:122;:::o"
          },
          "legacyAssembly": {},
        "metadata": "{\"compiler\":{\"version\":\"0.8.7+commit.e28d00a7\"},\"language\":\"Solidity\",\"output\":{\"abi\":[{\"inputs\":[],\"name\":\"retrieve\",\"outputs\":[{\"internalType\":\"uint256\",\"name\":\"\",\"type\":\"uint256\"}],\"stateMutability\":\"view\",\"type\":\"function\"},{\"inputs\":[{\"internalType\":\"uint256\",\"name\":\"num\",\"type\":\"uint256\"}],\"name\":\"store\",\"outputs\":[],\"stateMutability\":\"nonpayable\",\"type\":\"function\"}],\"devdoc\":{\"details\":\"Store & retrieve value in a variable\",\"kind\":\"dev\",\"methods\":{\"retrieve()\":{\"details\":\"Return value \",\"returns\":{\"_0\":\"value of 'number'\"}},\"store(uint256)\":{\"details\":\"Store value in variable\",\"params\":{\"num\":\"value to store\"}}},\"title\":\"Storage\",\"version\":1},\"userdoc\":{\"kind\":\"user\",\"methods\":{},\"version\":1}},\"settings\":{\"compilationTarget\":{\"contracts/1_Storage.sol\":\"Storage\"},\"evmVersion\":\"london\",\"libraries\":{},\"metadata\":{\"bytecodeHash\":\"ipfs\"},\"optimizer\":{\"enabled\":false,\"runs\":200},\"remappings\":[]},\"sources\":{\"contracts/1_Storage.sol\":{\"keccak256\":\"0xb6ee9d528b336942dd70d3b41e2811be10a473776352009fd73f85604f5ed206\",\"license\":\"GPL-3.0\",\"urls\":[\"bzz-raw://fe52c6e3c04ba5d83ede6cc1a43c45fa43caa435b207f64707afb17d3af1bcf1\",\"dweb:/ipfs/QmawU3NM1WNWkBauRudYCiFvuFE1tTLHB98akyBvb9UWwA\"]}},\"version\":1}",
        "storageLayout": {
          "storage": [
            {
              "astId": 4,
              "contract": "contracts/1_Storage.sol:Storage",
              "label": "number",
              "offset": 0,
              "slot": "0",
              "type": "t_uint256"
            }
          ],
          "types": { "t_uint256": { "encoding": "inplace", "label": "uint256", "numberOfBytes": "32" } }
        },
        "userdoc": { "kind": "user", "methods": {}, "version": 1 }
      }
    },
  },
  "sources": { "contracts/1_Storage.sol": { "id": 0 } }
}

We are only interested in the compilationTarget (e.g. contracts/1_Storage.sol and Storage) and not all other contracts provided. The two fields of our interest are evm.bytecode.object and evm.deployedBytecode.object.

Onchain vs. Recompiled Bytecodes​

Essentially we need to compare some onchain data against a compilation. We will find the onchain bytecodes using the chainId + address provided through an RPC. The recompiled bytecodes will come from the compilation output above.

Runtime vs. Creation Bytecodes​

We talked about the onchain and recompiled bytecodes. In addition to this, there are also two associated bytecode types to a contract we can perform the verification on.

Runtime bytecode is the bytecode that gets stored at the contract's address and that will be executed when this contract is called.

• The recompiled runtime bytecode is found under evm.deployedBytecode field of the compiler outputs.
• The onchain runtime bytecode is quite straightforward to get. It can be obtained with provider.getCode(0xabc..def), essentially with the eth_getCode RPC call to a node running the chain we are interested in with the chainId.

Creation bytecode or sometimes referred to as the "initcode" is the code that will be executed by the EVM to deploy this contract.

• The recompiled creation bytecode is found under evm.bytecode field of the compiler outputs.
• The onchain creation bytecode is a little trickier.
  • Typically, when a contract is being created by an EOA the receiver of the transaction is set to zero (tx.to=null), and the creation bytecode is placed in the transaction payload (tx.input or tx.data). The EVM interprets this as a contract creation and executes the transaction payload.
  • If a contract is created by another contract (factory pattern), we are going to have to look into the transaction traces, ie. every single step within the transaction execution to find the exact place where this contract's creation code was executed. Unfortunately this data is not easily available from RPCs like the runtime bytecode.

To be able to say "Yes this source code is the source code of this contract 0x48A331150C1b442444F4f0371a4daC9Ab2FC837D on Ethereum Holesky Testnet", we will compare the following and try to get a match:

• onchain vs. recompiled runtime bytecodes
• onchain vs. recompiled creation bytecodes

Matching and Transformations​

Now we are comparing the bytecodes.

If you're lucky you get a 100% match and you're done. Take this contract:

// SPDX-License-Identifier: GPL-3.0

pragma solidity >=0.8.2 <0.9.0;

contract Storage {

    uint256 number;

    function store(uint256 num) public {
        number = num;
    }

    function retrieve() public view returns (uint256){
        return number;
    }
}

{
  "compiler": {
    "version": "0.8.26+commit.8a97fa7a"
  },
  "language": "Solidity",
  "output": {
    "abi": [
      {
        "inputs": [
          
        ],
        "name": "retrieve",
        "outputs": [
          {
            "internalType": "uint256",
            "name": "",
            "type": "uint256"
          }
        ],
        "stateMutability": "view",
        "type": "function"
      },
      {
        "inputs": [
          {
            "internalType": "uint256",
            "name": "num",
            "type": "uint256"
          }
        ],
        "name": "store",
        "outputs": [
          
        ],
        "stateMutability": "nonpayable",
        "type": "function"
      }
    ],
    "devdoc": {
      "kind": "dev",
      "methods": {
        
      },
      "version": 1
    },
    "userdoc": {
      "kind": "user",
      "methods": {
        
      },
      "version": 1
    }
  },
  "settings": {
    "compilationTarget": {
      "contracts/1_Storage.sol": "Storage"
    },
    "evmVersion": "cancun",
    "libraries": {
      
    },
    "metadata": {
      "bytecodeHash": "ipfs"
    },
    "optimizer": {
      "enabled": false,
      "runs": 200
    },
    "remappings": [
      
    ]
  },
  "sources": {
    "contracts/1_Storage.sol": {
      "keccak256": "0x37ee358e0c9d3c9a75b75a2723ad8ab652c9e93ca38954426a5e9f8b80b83452",
      "license": "GPL-3.0",
      "urls": [
        "bzz-raw://2c734f79c940178a38088a7803161d0cc793ec51e917ce17fe279739368c0018",
        "dweb:/ipfs/QmRsfA8ZQo8TrnSKamMSC3JGAZtuNJhhuadHntY7AjEqew"
      ]
    }
  },
  "version": 1
}

Deployed at 0x48A331150C1b442444F4f0371a4daC9Ab2FC837D on Holesky Testnet, it has the following bytecodes:

Runtime Bytecodes: Onchain vs. Recompiled:

0x608060405234801561000f575f80fd5b5060043610610034575f3560e01c80632e64cec1146100385780636057361d14610056575b5f80fd5b610040610072565b60405161004d919061009b565b60405180910390f35b610070600480360381019061006b91906100e2565b61007a565b005b5f8054905090565b805f8190555050565b5f819050919050565b61009581610083565b82525050565b5f6020820190506100ae5f83018461008c565b92915050565b5f80fd5b6100c181610083565b81146100cb575f80fd5b50565b5f813590506100dc816100b8565b92915050565b5f602082840312156100f7576100f66100b4565b5b5f610104848285016100ce565b9150509291505056fea264697066735822122083bf1f648673d1110ee286ae659cc9c1299ab1d07bf83c96a591efa5df644f4864736f6c634300081a0033

0x608060405234801561000f575f80fd5b5060043610610034575f3560e01c80632e64cec1146100385780636057361d14610056575b5f80fd5b610040610072565b60405161004d919061009b565b60405180910390f35b610070600480360381019061006b91906100e2565b61007a565b005b5f8054905090565b805f8190555050565b5f819050919050565b61009581610083565b82525050565b5f6020820190506100ae5f83018461008c565b92915050565b5f80fd5b6100c181610083565b81146100cb575f80fd5b50565b5f813590506100dc816100b8565b92915050565b5f602082840312156100f7576100f66100b4565b5b5f610104848285016100ce565b9150509291505056fea264697066735822122083bf1f648673d1110ee286ae659cc9c1299ab1d07bf83c96a591efa5df644f4864736f6c634300081a0033

Creation Bytecodes: Onchain vs. Recompiled:

0x6080604052348015600e575f80fd5b506101438061001c5f395ff3fe608060405234801561000f575f80fd5b5060043610610034575f3560e01c80632e64cec1146100385780636057361d14610056575b5f80fd5b610040610072565b60405161004d919061009b565b60405180910390f35b610070600480360381019061006b91906100e2565b61007a565b005b5f8054905090565b805f8190555050565b5f819050919050565b61009581610083565b82525050565b5f6020820190506100ae5f83018461008c565b92915050565b5f80fd5b6100c181610083565b81146100cb575f80fd5b50565b5f813590506100dc816100b8565b92915050565b5f602082840312156100f7576100f66100b4565b5b5f610104848285016100ce565b9150509291505056fea264697066735822122083bf1f648673d1110ee286ae659cc9c1299ab1d07bf83c96a591efa5df644f4864736f6c634300081a0033

0x6080604052348015600e575f80fd5b506101438061001c5f395ff3fe608060405234801561000f575f80fd5b5060043610610034575f3560e01c80632e64cec1146100385780636057361d14610056575b5f80fd5b610040610072565b60405161004d919061009b565b60405180910390f35b610070600480360381019061006b91906100e2565b61007a565b005b5f8054905090565b805f8190555050565b5f819050919050565b61009581610083565b82525050565b5f6020820190506100ae5f83018461008c565b92915050565b5f80fd5b6100c181610083565b81146100cb575f80fd5b50565b5f813590506100dc816100b8565b92915050565b5f602082840312156100f7576100f66100b4565b5b5f610104848285016100ce565b9150509291505056fea264697066735822122083bf1f648673d1110ee286ae659cc9c1299ab1d07bf83c96a591efa5df644f4864736f6c634300081a0033

You'll see both the runtime and creation bytecodes match between the onchain and recompiled ones. So we say that the source-code of the contract 0x48A331150C1b442444F4f0371a4daC9Ab2FC837D on the Holesky Testnet is the one above.

But most of the times this is not as straightforward.

A lot of the times, parts of the bytecode are modified when the contract is being deployed. This is what we refer to as "transformations" within Sourcify and the Verifier Alliance. There are certain patterns in Solidity and Vyper contracts that can change between the recompiled vs. onchain bytecodes but even with different values, the contract behaves the same. You can guess that these transformations are "data" or "metadata" of the contract embedded in its bytecode, and not the functional bytecodes i.e. opcodes.

In the Verifier Alliance's json-schemas we define the possible transformations for the creation bytecode and the runtime bytecode as such:

Creation Bytecode Transformations​

There are 3 transformations possible as creation_transformations:

• constructorArguments
• cborAuxdata
• library

Constructor Arguments

The constructor arguments of a contract are ABI-encoded and appended at the end of the onchain creation bytecode.

As constructor arguments are only appended to the compiled bytecode, this is an insert (append) type transformation.

// SPDX-License-Identifier: GPL-3.0

pragma solidity >=0.7.0 <0.9.0;

contract Owner {

    address private owner;

    event OwnerSet(address indexed oldOwner, address indexed newOwner);

    modifier isOwner() {
        require(msg.sender == owner, "Caller is not owner");
        _;
    }
    constructor(address passedOwner) {
        owner = passedOwner; // 'msg.sender' is sender of current call, contract deployer for a constructor
        emit OwnerSet(address(0), owner);
    }

    function changeOwner(address newOwner) public isOwner {
        require(newOwner != address(0), "New owner should not be the zero address");
        emit OwnerSet(owner, newOwner);
        owner = newOwner;
    }

    function getOwner() external view returns (address) {
        return owner;
    }
} 

0x608060405234801561000f575f80fd5b506040516105c13803806105c18339818101604052810190610031919061014d565b805f806101000a81548173ffffffffffffffffffffffffffffffffffffffff021916908373ffffffffffffffffffffffffffffffffffffffff1602179055505f8054906101000a900473ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff165f73ffffffffffffffffffffffffffffffffffffffff167f342827c97908e5e2f71151c08502a66d44b6f758e3ac2f1de95f02eb95f0a73560405160405180910390a350610178565b5f80fd5b5f73ffffffffffffffffffffffffffffffffffffffff82169050919050565b5f61011c826100f3565b9050919050565b61012c81610112565b8114610136575f80fd5b50565b5f8151905061014781610123565b92915050565b5f60208284031215610162576101616100ef565b5b5f61016f84828501610139565b91505092915050565b61043c806101855f395ff3fe608060405234801561000f575f80fd5b5060043610610034575f3560e01c8063893d20e814610038578063a6f9dae114610056575b5f80fd5b610040610072565b60405161004d919061028e565b60405180910390f35b610070600480360381019061006b91906102d5565b610099565b005b5f805f9054906101000a900473ffffffffffffffffffffffffffffffffffffffff16905090565b5f8054906101000a900473ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff163373ffffffffffffffffffffffffffffffffffffffff1614610126576040517f08c379a000000000000000000000000000000000000000000000000000000000815260040161011d9061035a565b60405180910390fd5b5f73ffffffffffffffffffffffffffffffffffffffff168173ffffffffffffffffffffffffffffffffffffffff1603610194576040517f08c379a000000000000000000000000000000000000000000000000000000000815260040161018b906103e8565b60405180910390fd5b8073ffffffffffffffffffffffffffffffffffffffff165f8054906101000a900473ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff167f342827c97908e5e2f71151c08502a66d44b6f758e3ac2f1de95f02eb95f0a73560405160405180910390a3805f806101000a81548173ffffffffffffffffffffffffffffffffffffffff021916908373ffffffffffffffffffffffffffffffffffffffff16021790555050565b5f73ffffffffffffffffffffffffffffffffffffffff82169050919050565b5f6102788261024f565b9050919050565b6102888161026e565b82525050565b5f6020820190506102a15f83018461027f565b92915050565b5f80fd5b6102b48161026e565b81146102be575f80fd5b50565b5f813590506102cf816102ab565b92915050565b5f602082840312156102ea576102e96102a7565b5b5f6102f7848285016102c1565b91505092915050565b5f82825260208201905092915050565b7f43616c6c6572206973206e6f74206f776e6572000000000000000000000000005f82015250565b5f610344601383610300565b915061034f82610310565b602082019050919050565b5f6020820190508181035f83015261037181610338565b9050919050565b7f4e6577206f776e65722073686f756c64206e6f7420626520746865207a65726f5f8201527f2061646472657373000000000000000000000000000000000000000000000000602082015250565b5f6103d2602883610300565b91506103dd82610378565b604082019050919050565b5f6020820190508181035f8301526103ff816103c6565b905091905056fea2646970667358221220f875573b100fa5b8ca92df9690b6cf74e0fa0ecbd95165b3e274ee6e68c27e9664736f6c634300081a0033

0x608060405234801561000f575f80fd5b506040516105c13803806105c18339818101604052810190610031919061014d565b805f806101000a81548173ffffffffffffffffffffffffffffffffffffffff021916908373ffffffffffffffffffffffffffffffffffffffff1602179055505f8054906101000a900473ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff165f73ffffffffffffffffffffffffffffffffffffffff167f342827c97908e5e2f71151c08502a66d44b6f758e3ac2f1de95f02eb95f0a73560405160405180910390a350610178565b5f80fd5b5f73ffffffffffffffffffffffffffffffffffffffff82169050919050565b5f61011c826100f3565b9050919050565b61012c81610112565b8114610136575f80fd5b50565b5f8151905061014781610123565b92915050565b5f60208284031215610162576101616100ef565b5b5f61016f84828501610139565b91505092915050565b61043c806101855f395ff3fe608060405234801561000f575f80fd5b5060043610610034575f3560e01c8063893d20e814610038578063a6f9dae114610056575b5f80fd5b610040610072565b60405161004d919061028e565b60405180910390f35b610070600480360381019061006b91906102d5565b610099565b005b5f805f9054906101000a900473ffffffffffffffffffffffffffffffffffffffff16905090565b5f8054906101000a900473ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff163373ffffffffffffffffffffffffffffffffffffffff1614610126576040517f08c379a000000000000000000000000000000000000000000000000000000000815260040161011d9061035a565b60405180910390fd5b5f73ffffffffffffffffffffffffffffffffffffffff168173ffffffffffffffffffffffffffffffffffffffff1603610194576040517f08c379a000000000000000000000000000000000000000000000000000000000815260040161018b906103e8565b60405180910390fd5b8073ffffffffffffffffffffffffffffffffffffffff165f8054906101000a900473ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff167f342827c97908e5e2f71151c08502a66d44b6f758e3ac2f1de95f02eb95f0a73560405160405180910390a3805f806101000a81548173ffffffffffffffffffffffffffffffffffffffff021916908373ffffffffffffffffffffffffffffffffffffffff16021790555050565b5f73ffffffffffffffffffffffffffffffffffffffff82169050919050565b5f6102788261024f565b9050919050565b6102888161026e565b82525050565b5f6020820190506102a15f83018461027f565b92915050565b5f80fd5b6102b48161026e565b81146102be575f80fd5b50565b5f813590506102cf816102ab565b92915050565b5f602082840312156102ea576102e96102a7565b5b5f6102f7848285016102c1565b91505092915050565b5f82825260208201905092915050565b7f43616c6c6572206973206e6f74206f776e6572000000000000000000000000005f82015250565b5f610344601383610300565b915061034f82610310565b602082019050919050565b5f6020820190508181035f83015261037181610338565b9050919050565b7f4e6577206f776e65722073686f756c64206e6f7420626520746865207a65726f5f8201527f2061646472657373000000000000000000000000000000000000000000000000602082015250565b5f6103d2602883610300565b91506103dd82610378565b604082019050919050565b5f6020820190508181035f8301526103ff816103c6565b905091905056fea2646970667358221220f875573b100fa5b8ca92df9690b6cf74e0fa0ecbd95165b3e274ee6e68c27e9664736f6c634300081a00330000000000000000000000002ccbc64dd59efccb5a9129b412f4ea2ef5dd1b99

...e9664736f6c634300081a0033

...e9664736f6c634300081a00330000000000000000000000002ccbc64dd59efccb5a9129b412f4ea2ef5dd1b99

0000000000000000000000002ccbc64dd59efccb5a9129b412f4ea2ef5dd1b99

CBOR Auxdata

By default both the Solidity and Vyper compilers add some metadata to contract's bytecode. This is written in CBOR encoding in bytes. For Solidity contracts you can check playground.sourcify.dev

This section does not contain any opcodes or executable bytes and therefore independent from the source code we are trying to verify against. So two contracts with different cborAuxdata sections but with the rest of the bytecodes matching should match. However, Solidity and recently Vyper place an integrity hash within this cbor encoded section. If in addition to the rest of the bytecode the integrity hashes match too, this means the compilation is 100% identical with the original one, not even a whitespace or a comment in the source code is different.

However it is not straightforward to find where these sections are in the bytecode. You can read our blog post for more technical info how to do this.

To read more about how this helps with verification see our docs

Since we replace a cborAuxdata with another, this is a replace type transformation.

// SPDX-License-Identifier: GPL-3.0

pragma solidity >=0.8.2 <0.9.0;

contract Storage {

    uint256 number;

    function store(uint256 num) public {
        number = num;
    }

    function retrieve() public view returns (uint256){
        return number;
    }
}

contract Storage {
    // This is a comment
    uint256 number;

0x608060405234801561000f575f80fd5b5060043610610034575f3560e01c80632e64cec1146100385780636057361d14610056575b5f80fd5b610040610072565b60405161004d919061009b565b60405180910390f35b610070600480360381019061006b91906100e2565b61007a565b005b5f8054905090565b805f8190555050565b5f819050919050565b61009581610083565b82525050565b5f6020820190506100ae5f83018461008c565b92915050565b5f80fd5b6100c181610083565b81146100cb575f80fd5b50565b5f813590506100dc816100b8565b92915050565b5f602082840312156100f7576100f66100b4565b5b5f610104848285016100ce565b9150509291505056fea264697066735822122083bf1f648673d1110ee286ae659cc9c1299ab1d07bf83c96a591efa5df644f4864736f6c634300081a0033

0x608060405234801561000f575f80fd5b5060043610610034575f3560e01c80632e64cec1146100385780636057361d14610056575b5f80fd5b610040610072565b60405161004d919061009b565b60405180910390f35b610070600480360381019061006b91906100e2565b61007a565b005b5f8054905090565b805f8190555050565b5f819050919050565b61009581610083565b82525050565b5f6020820190506100ae5f83018461008c565b92915050565b5f80fd5b6100c181610083565b81146100cb575f80fd5b50565b5f813590506100dc816100b8565b92915050565b5f602082840312156100f7576100f66100b4565b5b5f610104848285016100ce565b9150509291505056fea2646970667358221220a78868ca872a9b18cf3baa44c67d38a41d3c4e8fa71a98b42510b47c5e2a8a2764736f6c634300081a0033

...50509291505056fea264697066735822122083bf1f648673d1110ee286ae659cc9c1299ab1d07bf83c96a591efa5df644f4864736f6c634300081a0033

...50509291505056fea2646970667358221220a78868ca872a9b18cf3baa44c67d38a41d3c4e8fa71a98b42510b47c5e2a8a2764736f6c634300081a0033

Libraries

Libraries (in Solidity) are contracts that are deployed once to an address and can be used multiple times. If a contract is using a deployed library, the address of this library is embedded inside bytecode.

The process of passing the deployed library's address to the contract being compiled is called "library linking". Normally, this can be done by passing the libraries field to the compiler or with the --libraries flag. Otherwise the compiler will put placeholders within the bytecode where the deployed library addresses will be placed.

The placeholders look like this:

__$53aea86b7d70b31448b230b20ae141a537$__

So after a compilation, if a contract has unlinked libraries, these placeholders will be replaced with a contract address. This is also a replace type transformation.

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

library Lib {
    function sum(uint256 a, uint256 b) external pure returns (uint256) {
        return a + b;
    }
}
 
contract A {
    function sum(uint256 a, uint256 b) external pure returns (uint256) {
        return Lib.sum(a, b);
    }
}

....0910390f35b5f73__$c6b2492631a03fef20e8d24218f6bc9947$__63cad089...

...0910390f35b5f7399f7c0086ab897c8fe65ebff268c732a7e15b25f63cad089...

Runtime Bytecode Transformations​

We talked about what transformation can be done to the creation bytecodes. The transformations on runtime bytecode are similar to the creation bytecode but with two differences.

First, since constructor arguments can only exist for the creation bytecode, the runtime bytecode does not have a constructorArguments type transformation.

Second, in addition the runtime bytecode can have an immutable transformation.

The library and cborAuxdata transformations for the runtime and creation bytecodes are exactly the same.

Immutables

Immutables are contract variables that can only be set at the deploy time, cannot be changed, and embeded within the contract's bytecode itself, in contrast to other variables persisted in the contract storage.

Solidity and Vyper compilers handle immutables differently. For Vyper contracts, the immutable variables are always at the end of the runtime bytecode, so these are appened. Therefore, Vyper contracts will have an insert type transformation. In Solidity, immutables can be anywhere in the bytecode but their positions will be output within the immutableReferences field. Using this compiler output, we can apply the transformation on the recompiled runtime bytecode and see if it matches the onchain runtime bytecode.

pragma solidity >=0.7.0;

contract WithImmutables {
    uint256 public immutable _a;
    
    string _name;
    
    constructor (uint256 a) {
        _a = a;
    }
    
    function sign(string memory name) public {
        _name = name;
    }
    
    function read() public view returns(string memory) {
        return _name;
    }
}

...e3be2f4c80faf25310b966a08079e44064736f6c634300081a003300000000000000000000000000000000000000000000000000000000000000ff

...059f565b5050565b7f000000000000000000000000000000000000000000000000000000000000000081565b5f8151...

...059f565b5050565b7f00000000000000000000000000000000000000000000000000000000000000ff81565b5f8151...

# pragma version ^0.4.0

OWNER: public(immutable(address))
MY_IMMUTABLE: public(immutable(uint256))

@deploy
def __init__(val: uint256):
    OWNER = msg.sender
    MY_IMMUTABLE = val


@external
@view
def get_my_immutable() -> uint256:
  return MY_IMMUTABLE

...0657283000400001400000000000000000000000000000000000000000000000000000000000000ff

...ffd5b5f80fd001800360054

...ffd5b5f80fd0018003600540000000000000000000000002ccbc64dd59efccb5a9129b412f4ea2ef5dd1b9900000000000000000000000000000000000000000000000000000000000000ff

Call Protection

Since libraries are not meant to be called with CALL instead of DELEGATECALL or CALLCODE (except view and pure functions), the Solidity compiler places a check for who's calling in the beginning of the library bytecodes. This callProtection starts with 0x73 (PUSH20) followed by the address of the contract itself and checks the current address against this.

At the deploy time, these 20 bytes will be replaced with the contract's own address.

See Solidity docs.

0x7300000000000000000000000000000000000000003014608060405260043610610034575f3560e01c8063cad0899b14610038575b5f80fd5b610052600480360381019061004d91906100b4565b610068565b60405161005f9190610101565b60405180910390f35b5f81836100759190610147565b905092915050565b5f80fd5b5f819050919050565b61009381610081565b811461009d575f80fd5b50565b5f813590506100ae8161008a565b92915050565b5f80604083850312156100ca576100c961007d565b5b5f6100d7858286016100a0565b92505060206100e8858286016100a0565b9150509250929050565b6100fb81610081565b82525050565b5f6020820190506101145f8301846100f2565b92915050565b7f4e487b71000000000000000000000000000000000000000000000000000000005f52601160045260245ffd5b5f61015182610081565b915061015c83610081565b92508282019050808211156101745761017361011a565b5b9291505056fea26469706673582212209d56669aec61ac1867a59a6d0472bd74975630174bc6e74d7e6881bba174c30564736f6c634300081a0033

0x7399f7c0086ab897c8fe65ebff268c732a7e15b25f3014608060405260043610610034575f3560e01c8063cad0899b14610038575b5f80fd5b610052600480360381019061004d91906100b4565b610068565b60405161005f9190610101565b60405180910390f35b5f81836100759190610147565b905092915050565b5f80fd5b5f819050919050565b61009381610081565b811461009d575f80fd5b50565b5f813590506100ae8161008a565b92915050565b5f80604083850312156100ca576100c961007d565b5b5f6100d7858286016100a0565b92505060206100e8858286016100a0565b9150509250929050565b6100fb81610081565b82525050565b5f6020820190506101145f8301846100f2565b92915050565b7f4e487b71000000000000000000000000000000000000000000000000000000005f52601160045260245ffd5b5f61015182610081565b915061015c83610081565b92508282019050808211156101745761017361011a565b5b9291505056fea26469706673582212209d56669aec61ac1867a59a6d0472bd74975630174bc6e74d7e6881bba174c30564736f6c634300081a0033

After matching​

If we got a match, it means that the provided source code and metadata can be associated with the contract at this chainId and address and we mark this contract as verified. It also means we can start from recompiled bytecodes, apply each runtime or creation transformations on top of the respective recompiled bytecodes, and obtain the onchain bytecodes.

Finally, we store our contract in our storage backend of choice. Currently this can be a file system or a SQL database in Sourcify's case. By default we do both. See the Contract Repository docs for more information.

EOF and the road ahead​

Big changes to how the EVM bytecode is structured is coming up with the EVM Object Format proposal. According to how the compilers implement these changes, the verification process face changes and verifying pre-EOF contracts will be different than the EOF contracts.

One nice feature of the EOF is it separates code from data and gives structure to code. This makes us verifiers' lives much easier compared to the completely unstructured legacy EVM code. The legacy EVM example can be seen in our post Finding Auxdatas in the Bytecode. However as of the current specification, the metadata is still not fully isolated from the contract code as there is no separate section for that information. Current compiler implementations put the metadata in the data_section of EOF and this unfortunately does affect the contract's code. That's why we are proposing a separate metadata_section in the EVM Object Format that cannot be reached by the EVM and any changes to that will not affect the contract's code. If this is implemented, verifiers can completely ignore the metadata parts of a container, and won't have to look for workarounds.

You can check our proposal EIP-7834

The problem​

Source code verification requires compiling a contract written in a high-level language (e.g. Solidity, Vyper) to the bytecode, and comparing the compiled bytecode with the onchain bytecode. If there’s a match, we can say the given high-level code is the source-code of the contract at the given address.

The runtime bytecode of contracts by default also contain a special field at the end in CBOR encoding (auxdata). This field contains the hash of the contract metadata file (metadata hash), which acts as a fingerprint of the compilation. The metadata file has compiler settings, and source file hashes so the slightest change in the compiler settings or even a whitespace in any of the source files will cause a change in the metadata hash.

For a visual explanation of everything above, check out playground.sourcify.dev

Because of its sensitivity, some verifiers leave this field out in verification. In Sourcify’s case, if the recompiled bytecode and the onchain bytecodes match each other exactly (including the auxdata), it’s great. This will give us a “full match”. If not, we need to find the auxdatas and leave them out when comparing to be able to get at least a "partial match".

However this is not always trivial especially in these cases:

1. The creation bytecode of a contract does not necessarily have the CBOR encoded part at the very end of the bytecode. Although sometimes it’s found there, this field can be anywhere. In fact the only reason the CBOR encoded part is in the creation bytecode is because the runtime bytecode is embedded inside the creation bytecode as a whole.

   When executing the creation bytecode i.e. deploying the contract, the contract’s runtime bytecode needs to be returned. The runtime bytecode is already inside the creation bytecode so this part is extracted and returned by taking the offset and the length for the related bytecode and returning it. This can be anywhere inside the code. (Check this article for a comprehensive deep dive into contract creation)

2. The runtime bytecode has the CBOR encoded part always at the end of the contract (unless turned off with appendCbor: false). But the bytecode can contain other contract bytecodes nested inside, which also can have their own auxdatas, and these parts need to be ignored for a verification. This is found for example in factory contracts where a contract creates another contract and the child contract’s code is nested in the factory’s bytecode.

Now for other “special” parts of the bytecode, the compiler outputs the positions such as immutables in immutableReferences. Unfortunately this is not the case for auxdatas and we need to look elsewhere and find workarounds.

Workarounds​

If not the exact positions of the auxdatas, the compiler at least outputs the values. Inside the legacyAssembly object of the compiler output we can find the auxdata, which is under the key .auxdata

example legacyAssembly:

{
    ".code": [],
    ".data": {
        "0": {
            ".auxdata": "a26469706673582212203a05097003697b26b1da819218bcd95779598eaa90539e82a59ecbe4c09757e364736f6c63430007000033",
            ".code": [...]
        }
    }
}

At this point, one could think to do a simple string search in the bytecode for the auxdatas found in legacyAssembly, but it would be possible for an attacker to trick the search function and falsely ignore parts of the bytecode that are not supposed to be ignored.

The vulnerability​

Imagine we have the auxdata string from the compiler’s legacyAssembly above.

a26469706673582212203a05097003697b26b1da819218bcd95779598eaa90539e82a59ecbe4c09757e364736f6c63430007000033

This could be the auxdata of a simple child contract inside the whole bytecode that we know won’t be affected by the changes of our main contract.

For this specific example the attacker could embed these bytes inside the bytecode such a code in the main contract:

assembly {
    // Split the code from a push opcode:
    // a26469706673582212203a05097003697b26b1da819218bcd957
    // 79 (PUSH26)
    // 598eaa90539e82a59ecbe4c09757e364736f6c63430007000033

    mstore(0x598eaa90539e82a59ecbe4c09757e364736f6c63430007000033, 0xa26469706673582212203a05097003697b26b1da819218bcd957)
    // PUSH26 0xa26469706673582212203a05097003697b26b1da819218bcd957
    // PUSH26 0x598eaa90539e82a59ecbe4c09757e364736f6c63430007000033
    // MSTORE
}

By chance (really) this auxdata of 53 bytes is split into two exactly from the middle but this doesn’t have to be the case. Remember the large middle portion of the CBOR encoding contains the IPFS hash so one can salt and iterate it.

Imagine the source code of the attacker compiles to the code below. Putting new lines to demonstrate the (allegedly) auxdata part:

0x6080...b732b960691b604482015260640160405180910390fd5b5f80546040516001600160a01b03808516939216917f342827c97908e5e2f71151c08502a66d44b6f758e3ac2f1de95f02eb95f0a73591a35f80546001600160a01b0319166001600160a01b039290921691909117905579
  a26469706673582212203a05097003697b26b1da819218bcd95779598eaa90539e82a59ecbe4c09757e364736f6c63430007000033
  52565b5f6a636f6e736f6c652e6c6f6790505f80835160208501845afa505050565b6101756101a4565b565b5f60208284031215610187575f80fd5b81356001600160a01b038116811461019d575f80fd5b9392505050565b634e487b7160e01b5f52605160045260245ffdfea2646970667358221220212c0514e8c0db310d02690fc2def199f4fba3828f2401ec2b8d7104e450b8b164736f6c63430008180033

This is what we get from the source code the attacker gives us to verify. So we go: “Oh right there's an auxdata a26469706673582212203a05097003697b26b1da819218bcd95779598eaa90539e82a59ecbe4c09757e364736f6c63430007000033 in this bytecode. We should ignore the corresponding part in the (onchain) bytecode to have a partial match.”

Oops now we are ignoring a part in the bytecode that we're not supposed to. These code parts are only meant for non-executable code whereas we embedded this with an assembly block.

In the attacker’s onchain bytecode (what actually will be executed vs. the verified code) the attacker could have placed anything in this assembly block for 53 bytes. I leave it up to your imagination what can be done with this ignored bytecode block.

The gist is, we need to make sure these to-be-ignored blocks are actually auxdatas and not coming for an executable code block. How do we do it?

The solution(s)​

Well, we know that the IPFS hash inside the auxdata is the hash of the metadata file and the metadata file contains the source file hashes. So we can touch all source files to change their hashes, e.g. by adding a whitespace at the end of each. By touching every single source file, we make sure the nested auxdatas will be modified as well. If we compile again, we will have the exact same bytecode just with differences at the metadata hashes. Then we can locate the metadata hashes by comparing the original and edited bytecodes side by side.

But we need one more thing: Now we know where the metadata hashes are but that is just a substring of the whole CBOR auxdata. So we need to figure out where the CBOR auxdata starts and ends.

Blockscout solution​

One way to do this is to start at the metadata hash positions we've found by comparing and go extend the byte substring byte-by-byte and each time try to decode the whole byte string in CBOR. If at one point successful, we know that the auxdata ends here. Remember that right after the CBOR encoding you'll find the length of the encoded part, so we know where it starts as well.

Indeed this is how Blockscout finds the auxdata positions.

Sourcify solution​

The way we approach this in Sourcify is by again making use of the legacyAssembly.

These are roughly the steps:

1. Use bytecodes: Compare the original bytecode to the whitespaced (edited) contract’s bytecode. This will give us the positions of the metadata hashes, remember not the whole auxdata.
2. Use legacyAssembly: Compare the auxdatas from legacyAssembly s of both contracts. We will get a auxdataDiff between each auxdata (1st auxdata in original vs 1st in edited etc.). The diff will not exactly be the whole metadata hashes because CIDv0 IPFS hashes start with Qm but the rest of the hash. The other parts of the auxdatas will be the same. We also keep the position of the diff inside the whole auxdata diffStart:

   interface AuxdataDiff {
       real: string;
       diffStart: number;
       diff: string;
   }
   
3. Remember these are the metadata hashes. If they are equal, we can now find where the whole auxdata starts with:

   for (const position of positions) {
       for (const auxdataDiff of auxdataDiffs) {
           // Compare if the diff from raw bytecode is equal the diff from `legacyAssembly` auxdatas
           if (editedBytecode.substring(position + auxdataDiff.diff.length) === auxdataDiff.diff)
               return originalBytecode.substring(position - auxdataDiff.diffStart, position + auxdataDiff.diff.length);
       }
   }
   

Original:

0x6080...                CBOR auxdata     
1909117905579a26469706673582212203a05097003697b26b1da819218bcd95779598eaa90539e82a59ecbe4c09757e364736f6c6343000700003352565b5f6a636f6e736f6c652e6c6

Edited:

0x6080...                CBOR auxdata     
1909117905579a2646970667358221220dceca8706b29e917dacf25fceef95acac8d90d765ac926663ce4096195952b6164736f6c6343000700003352565b5f6a636f6e736f6c652e6c6
             └──────────────────┘↑
                diffStart        position

An Alternative​

1. Start with a string search inside the bytecode for the auxdatas from legacyAssembly of the contract. Now we have the positions of potential auxdatas of the original contract.
2. Next we whitespace the source files and compile the contract again. Let’s call it the edited contract.
3. Finally we check if the bytecode substrings from the original contract and the edited contract have changed at the positions we found at the 1st step. We expect these to change if they indeed contain a real auxdata and not some custom bytecode.

Thanks to Rim from Blockscout for pointing out this alternative.

Making life easier for verifiers​

To avoid doing all these nitty workarounds we just proposed the Solidity compiler to output the positions of the auxdatas, similar to the immutableReferences field: https://github.com/ethereum/solidity/issues/14827

We are still going to need to do this for the compiler versions before this gets implemented but still it would be less work in verification, particularly not having to compile contracts twice.

Since we edited the original source code with whitespaces and compiled the contract, we also have the legacyAssembly for the edited contract, which contain auxdatas. If we compare all the auxdatas extracted from legacyAssembly s of both, we will get a diff of each auxdata field which will be the metadata hashes. The rest of the auxdatas will be the same.

Introduction​

Solidity compiler has a feature, not known by everyone, that appends the IPFS hash of the contract metadata to the contract bytecode. This hash effectively acts as a fingerprint of the compilation, and when deployed, goes onchain. With that, we can verify the contracts "perfectly" and fetch the contract source code from IPFS. One of our missions at Sourcify is to make this feature more known and used, but not everyone is a fan of it.

(If you don't fully understand the metadata hash check out our playground to see it in action.)

I argue this is the only foolproof way to verify contracts. Languages and tooling should come together and come up with a common standard. We should look back at what worked and what didn't, and come up with a better next version.

Runtime code vs Creation code​

In source-code verification you compare a bytecode to a high-level code (Solidity, Vyper).

When you compile a contract you get two bytecodes:

Runtime bytecode is the code of the contract living on the blockchain. This is what really gets executed when you call a contract. You'll find it if you look at the bytecode of an unverified contract in a block explorer or when you call eth_getCode(address) on the contract.

Creation bytecode is the code that will be executed by the EVM when the contract is being deployed, which will store the runtime code at contract's address.

Since the terms are not well defined, some terminology:

• "code" = "bytecode" in this context. Sometimes people just call it "runtime code", or "creation code".
• "Init code" = "Creation bytecode". This is usually used in create2 context.
• "Deployed Bytecode" = "Runtime Bytecode". This is another common way to refer to the runtime bytecode by the Solidity compiler and frameworks. I refrain from using this as sometimes the contract is not deployed and "runtime code" is more accurate.
• evm.bytecode = "Creation bytecode". The Solidity compiler refers to it as this in the output.
• evm.deployedBytecode = "Runtime bytecode". Same as above.

Which bytecode?​

Let's go back to the source code verification. The problem we are trying to solve is we have a contract, and we want to see the original source code of it. Because we humans, can't really read bytecodes.

However, a contract has two bytecodes, which one should we compare the source code to?

Verifying with Creation Bytecode​

One can say that the bytecode counterparty of a contract written in a high level language is the creation bytecode. Because, in a typical contract deployment this is what you give to the EVM to execute.

The problem with the creation bytecode is that it's not always stored onchain. The only time you see this is when you deploy a contract from an Externally Owned Account (EOA) by putting the creation bytecode in the tx.data and setting the receiver tx.to to null. In that case you'll see the creation bytecode if you look at the transaction.

However, for contracts created by other contracts (e.g. factories) it is executed once and then discarded. So someone needs to index and save the creation bytecodes somewhere and you need to trust them. Whereas the runtime bytecode is stored onchain and you can request it from your node with eth_getCode.

On the other hand, the creation bytecode of a contract is not necessarily what the compiler outputs. The creation bytecode can be any code that will execute and store the runtime bytecode at the contract address. See @ricmoo's CREATE2 example. He demonstrates how to deploy and SELFDESTRUCT a contract, and finally deploy a completely different contract at the same address, even though CREATE2 addreses depend on the init code. In this case the init code is the same but it dynamically gets and writes the contract code from somewhere else. If you change the code where it's dynamically fetched from, you deploy a different contract at the same address. So for this contract, even if we knew its original source code, we can't compile and compare against its creation code.

Verifying with the Runtime Bytecode​

The runtime bytecode is the actual code of the contract and is readily available at eth_getCode. The compiler also outputs the runtime bytecode so one can verify contracts with the runtime bytecode too. With that, you can easily verify a contract on the "edge" (i.e. on your machine) trustlessly by getting the bytecode from your execution client.

The compiler output can be different than the onchain one as during deployment the runtime bytecode can be modified by writing the immutable values and the linked libraries in the placeholders. It's ok because, for Solidity, the compiler outputs the immutableReferences and libraries have a __$ placeholder, so we know where these are positioned in the bytecode.

The problem is, not everything in high-level contract code is represented in the runtime bytecode. Imagine this contract excerpt:

    constructor() {
        owner = msg.sender;
        emit OwnerSet(address(0), owner);
    }

I can deploy this contract but verify it with a slightly different contract with the following constructor, which can have huge implications:

    constructor() {
        owner = tx.origin;
        emit OwnerSet(address(0), owner);
    }

This is because this constructor code part will not be included in the runtime bytecode, and the owner value is not stored inside the bytecode but in the contract's storage.

Verifying with the Runtime Bytecode + Metadata Hash​

There's a way around this problem. If you verify a contract with its metadata hash appended to the runtime bytecode, you'll get a full match. This means the source code you are looking at is exactly the same as the one that was originally compiled, because if you change anything about the contract (even a whitespace), the metadata hash will change and you will not get a "full match" but a "partial match".

This, I'd argue, is the only foolproof way to verify a contract's source code. This method covers all the cases above and the ones I haven't mentioned or we don't know about yet. By being based on the runtime code, this also removes the need to trust a third party to index the creation bytecode, and instead you can get the bytecode from your own execution client's JSON RPC interface.

Problems with the Metadata Hash​

The main critisism of this feature is that the hash is too sensitive. It's both a bug and a feature that the hash changes even with a whitespace change.

A bigger problem is with the paths of the .sources.

  ...
  "sources": {
    "myDirectory/myFile.sol": {
      "keccak256": "0x123...",
      "license": "MIT",
      "urls": [ "bzz-raw://7d7a...", "dweb:/ipfs/QmN..." ]
    }
  }

The keys here are actually not file paths but source-unit names, meaning they can be arbitrary strings. This is especially a problem for projects deploying with CREATE2, where the address of the contract depends on the init code. Any difference in "path" will be a different metadata hash --> diferent bytecode --> different contract address. As a result, most of them just turn off this feature.

It's a bigger problem if the same codebase does not compile to the same bytecode on different platforms. The differences caused by comments/whitespaces are not that big of a deal if we can verify contracts at the deployment pipeline i.e. right at the point when they are deployed. This also means we need to stop flattening contracts. Ideally you never drag and drop any files to a website, but use a verification plugin on your tooling (Foundry, Hardhat) or IDE (Remix). No medium size contract would manually be verified.

What would be a more clever way to do this? If we are able get this right, we solve most of the problems.

Conclusion​

The two bytecodes associated with a contract are not always sufficient to correctly verify a contract. The only foolproof and decentralized way to do it is to use the runtime bytecode with the metadata hash appended to it. I believe this needs to be the default way to verify contracts, and only when you can't do it (like this bug), you should fall back to the partial match. Although at Sourcify we base our verification on this, most of the ecosystem don't make the partial vs full match distinction or are just aware of it.

As an outcome of this article I'd really want to see:

1. Other cases where a runtime bytecode or creation bytecode fails to correctly verify a contract.
2. Counter-arguments to the usefulness of the metadata hash.
3. Clever ways to mitigate the problems with the metadata hash.
4. Languages other than Solidity adopting this feature, and coming up with a standard for it.

Do have anything to add for these points above? Please reach out to me on Twitter or add your remarks in the discussion issue for this article (I'll link). I'll also be updating this article with the feedback I get, and be linking to discussions. This will be a living document.

In an ecosystem with the core values of transparency, security, and trust (and trustlessness); it is expected from all contract developers to publish their source code. If you're even slightly familiar with Ethereum, there is no need for further explaination.

But if I give you a source code, how do you make sure the published source code really is the source code of the contract? That's where source code verification comes into play.

What is source code verification?​

First thing first, all the smart contracts on blockchain are stored in bytecode. Just like our physical machines that only speak bits and bytes, Ethereum Virtual Machine also only understands bytes. If you ask the Ethereum blockchain the code of a contract, you only get a byte string.

So, let's say I give you a contract in Solidity and claim that this is the code behind the contract at "0xabcdef...". To verify, you need to make sure this code compiles to the same bytecode as the claimed contract at "0xabcdef...". This is the basic idea behind the smart contract verification: we compile a contract and check if the bytecode matches the one on blockchain.

You have probably made use of contract verification before. For many users this is the green checkmark in Etherscan:

You see the green checkmark and you are happy!

But is it really exactly the same code that is deployed?

The answer is, you don't know 🤷

In fact, no one else would be able to know except the contract developer, and he/she can't really prove it. The reason is, when compiling the contract i.e. translating the human-readable source code (in Solidity or any other higher-level language) to machine-readable bytecode, some information is lost. These include internal variable names, internal function names, names of contracts etc.

So yes this is functionally the same code as deployed: it compiles to the same bytecode as the original mysterious source code 🕵.

And you might be thinking, sure this is good enough. But:

• Someone can insert misleading comments, (internal) function or variable names
• Whoever verifies a contract first is chosen as the matching result, not the "authentic" one
• We can't verify things other than the contract's code itself (i.e. metadata)

In fact when not verified properly, it is possible to inject code that would be shown in the verified source code.

Enough bad news... There's actually a way to verify Solidity contracts that would cryptographically ensure the exactness of the source files and it is already here: It's called Sourcify!

This way of verifying contracts is what we call a perfect verification, (in contrast to partial verification). This is enabled by the Solidity contract metadata, and that the hash of it is appended to the contract's bytecode. The metadata hash acts as a fingerprint of the whole compilation and with the information in the metadata file we can completely reproduce the contract compilation.

Contract Metadata​

The Solidity compiler by default appends some information to the contract's bytecode in CBOR encoding. This special field, I like referring to as auxdata, usually contains the "Solidity version", the "metadata hash", and occasionally the "experimental" flag. The encoded data and it's decoding looks like this:

You can actually inspect this field and see the decoding in action for any contract in playground.sourcify.dev.

To see how this cryptographically ensures the exactness of the source files we need to look into the contents of the metadata file. The metadata file is a JSON document that looks like this and contains information on two things:

1. How to interact with the contract: ABI, documentation
2. How to reproduce a contract compilation: compiler version and settings, source file information

The latter is the relevant field for our purposes. Specifically, the fact that the metadata file contains source file hashes. To illustrate this, let's walk through what happens when you compile a contract and what happens when you change a source file.

When you compile a contract, the compiler computes the hashes of the source files and embeds this information in the metadata file. On the right side, you see the relevant fields of the metadata file:

Then the compiler takes the hash of this whole file:

And encodes it in the auxdata at the end of the bytecode:

So if you were to decode the auxdata you'd see:

What happens when we change something in the source files? Say we change a variable name or a comment in the new MyContract-diff.sol file. In turn the hash of the file changes, as well as the hash in the metadata:

...and of course the hash of the metadata file changes:

...and the auxdata changes:

Sooo, if we match both the bytecode + the appended auxdata, we have byte-by-byte exactly the same source code and compilation settings of the original deployed contract. This is a perfect verification.

If the bytecode matches but not the auxdata (which includes the metadata hash), we have a partial verification.

Did you notice?​

If you are familiar with IPFS and paid attention, you might ask: Can't we already get everything from the bytecode itself?

And yes, if published on IPFS, you can actually fetch the source code from the bytecode of a contract, because all the information is already there:

• The metadata IPFS hash is appended to the bytecode so (if published) you can fetch the metadata file.
• The metadata file contains (alongside the normal keccak256) the IPFS hashes of the source files so you can fetch the complete source code from IPFS.

So there's only one thing that you need to do as a contract developer: Publish your source files and metadata on IPFS.

Why do you need verification then? Isn't the source file already out there?

Although unlikely since the compiler does it automatically, someone can change the auxdata of the contract before deploying it and show you a different random source code. We make sure it really is the same code by doing a whole recompilation of the provided files and comparing the resulting bytecodes. Plus, we share all verified contracts in our repository on IPFS to make sure it's available.

Conclusion​

Perfect verification enables more secure and transparent verification on contracts, as well as other useful things such as decoding tx's and enabling human-readable contract interactions, but this is a topic for another article.

Next level smart contract verification is already here. We just need to adopt this way of verifying contracts as a community. Obviously, we need a lot of tooling, integrations, and more awareness. Let's step up and make this the standard way of verifying contracts!

(This article is a summary of my recent talks about Sourcify. If you are interested in learning more, check out one of the latest talks)