How Shyft improves the Solidity Compiler

Hello, My name is Alex Binesh and I am a Blockchain engineer at Shyft Network.

This document will describe the changes applied to the Solidity (Solc) compiler by the Shyft development team in an effort to extend the functionality and improve the efficiency of the existing Solidity compiler performance and interface. The Solidity compiler is written in C/C++ and is the main tool for creating and compiling Ethereum Smart Contracts.

There are two main improvement sets:

1. New Opcodes, Pragmas and Math libraries

As part of the effort at extending the Solidity compiler features, the following three main enhancements have been implemented pending a final design approval.

1.1 Custom Shyft Opcodes.

These opcodes have been added to the existing compiler, and tested, as part of the effort to create capabilities for Shyft attestation mechanism, which relates to Shyft’s financial services network. They are compiled as part of a Solidity Smart Contract application, and are eventually executed at run-time by the Ethereum Virtual Machine (EVM).

1.1.1 getattest (uint64, uint64) return (uint64)

1.1.2 checkAttestValid (address, uint256) return (bool)

1.1.3 getrevoke(address, uint64)return (uint64)

1.1.4 topoint (address, uint64)return (uint64)

1.1.5 merkleprove (bytes memory, bytes32,bytes32) return (bool)

1.2 Custom Shyft Math Function

The below Math function has also been added. Its purpose is to conduct bounds check on the result of a run-time expression.

• MathSafeFunction (uint64, uint64)

1.3 Custom Shyft Pragma directive

The below Pragma directive has been added. This Pragma does not have a specific purpose as of yet, but may be used to signal the compiler to compile against some or all Shyft libraries or opcodes. At this point it simply sets a global flag which can be used for conditional compilation. Currently the only Pragma attribute is “substance”, but eventually other attributes can be implemented to signal various behaviors to the compile. The pragma substance currently sets a flag:

• pragma substance;

2. Improved Error and Warning handling by Solidity Compile

The Solc compiler often times generates numerous warnings and abstract error messages as part of the compilation process. Thus far there has been no option to suppress the warnings emanating from the compilation process, however as part of extending and improving the Solc compiler this feature has been introduced

Additionally, there are a large number of vague stack overflow related messages which are difficult to decipher, especially when coding complex Solidity applications. Shyft is making a concerted effort to identify and display the root cause of such messages, where possible. So far a large number of such Stack related errors have been identified and improved upon. This is an ongoing effort and will require the Solidity developer community’s input to identify and remedy such scenarios.

2.1 Warning Suppression Flag

Currently the Solc compiler generates numerous warning messages and in some cases the compilation process exits with exceptions due to the overwhelming number of warnings generated. As part of the Shyft Solc compiler enhancement, a flag has been introduced to provide Warning Message suppression.

The format is as follows: “solc — bin — no-warnings MyFile.solc” .

This would suppress all warnings emanating from the compiler while compiling Solidity codes.

Note: In Truffle and Remix-IDE this flag should be passed to the Soljson by setting the appropriate parameter(s). This feature is “work In progress” at Shyft.

2.2 Error Handling

2.2.1 Assembly Code Error handling

Scenario: In an assembly{} block in a Solidity application, if an invalid expression is entered, a generic message is generated:

“Error: Unbalanced stack at the end of a block: 1 surplus item(s)”

• Sample function to trigger this error

function unbalancedStack() {

uint iVar1; uint iVar2; uint iVar3;

assembly { iVar3}

}

New Error Message

Error: Unbalanced stack at the end of a block: 1 surplus item(s).

assembly {

^

Spanning multiple lines.

Error: Please check the expressions in your assembly code as one or more may be causing this error:

assembly { iVar3

2.2.2 Solidity Code Error handling

Currently there are many instances where in the course of compiling a Solidity application a generic error is generated, generally stating “Stack Too Deep, try removing local variables.” This is caused by too many items pushed onto the stack for a particular function, however no more detail is provided as to the root causes of the error.

In general the stack allows for a total of 16 items to be pushed onto it.

These items could be a combination of the number of:

• Local variable declaration
• Parameters passed into the function
• Parameters returned from a function
• Index references in an array

As part of the Shyft Solc error message enhancements, the error messages have been improved significantly to state the scenario triggering this error, and where possible to directly point to the potential excess variables or parameters utilized.

Though a task still in progress, thus far the following new messages are printing whereas before there was just the generic message “Stack too deep…” :

2.2.2.1 Scenario: A combination of more than 16 parameters declared and array indices are referenced in this function

Sample function to trigger this error

function getUser(){

int[] MyVariable1; int[] MyVariable2; int[] MyVariable3;

int[] MyVariable4; int[] MyVariable5; int[] MyVariable6; int[] MyVariable7;

int[] MyVariable8; int[] MyVariable9; int[] MyVariable10;

int[] MyVariable11; int[] MyVariable12; int[] MyVariable13;

int[] MyVariable14; int[] MyVariable15; int[] MyVariable16;

MyVariable1 [0] = 3; return (); }

New Error Message

Compiler error: Stack too deep, try removing local variables.

This could be because the combination of Returned parameters and Array index references add up to more than 16

Please see this line:

MyVariable1[0] = 3;

^ — — — — -^

NOTE: This error seems to only occur if the index referenced is in the first variable in the declaration list, in this case MyVariable1 [0] = 3;. Any reference to other variables would not trigger this error, e.g. MyVariable2 [0] = 3; would not trigger this error.

2.2.2.2 Scenario: More than 16 parameters declared in this function

Sample function to trigger this error

function getUser(){

int[] MyVariable1; int[] MyVariable2; int[] MyVariable3; int[] MyVariable4;

int[] MyVariable5; int[] MyVariable6; int[] MyVariable7; int[] MyVariable8;

int[] MyVariable9; int[] MyVariable10; int[] MyVariable11;

int[] MyVariable12; int[] MyVariable13; int[] MyVariable14;

int[] MyVariable15; int[] MyVariable16; int[] MyVariable17;

return ();

}

New Error Message

Compiler error: Stack too deep, try removing local variables.

You have declared more than 16 variables in this function:

function getUser()

^

Spanning multiple lines.

2.2.2.3 Scenario: More than 16 parameters passed to AND declared in this function

Sample function to trigger this error

function getUser(uint256 uuid1){

int[] MyVariable1; int[] MyVariable2; int[] MyVariable3; int[] MyVariable4;

int[] MyVariable5; int[] MyVariable6; int[] MyVariable7; int[] MyVariable8;

int[] MyVariable9; int[] MyVariable10; int[] MyVariable11;

int[] MyVariable12; int[] MyVariable13; int[] MyVariable14;

int[] MyVariable15; int[] MyVariable16;

return ();

}

New Error Message

Compiler error: Stack too deep, try removing local variables.

You cannot have more than a total of 16 Parameters passed to AND Declared in this function:

Possible Extra Parameters: uuid1

function getUser(uint256 uuid1)

^ — — ^

Spanning multiple lines

2.2.2.4 Scenario: More than 16 parameters returned from AND declared in this function

Sample function to trigger this error

function getUser() constant returns (int)

{

int[] MyVariable1; int[] MyVariable2; int[] MyVariable3; int[] MyVariable4; int[] MyVariable5; int[] MyVariable6; int[] MyVariable7; int[] MyVariable8;int[] MyVariable9; int[] MyVariable10; int[] MyVariable11; int[] MyVariable12; int[] MyVariable13; int[] MyVariable14;int[] MyVariable15; int MyVariable16; return (MyVariable16);

}

New Error Message

Compiler error: Stack too deep, try removing local variables.

This is caused probably because you have a total of more than 16 local variables declared in and Returned from this function

(int) {

^-^

2.2.2.5 Scenario: More than 16 parameters are returned from his function

Sample function to trigger this error

function getUser() constant returns

(int, int, int, int, int, int, int, int, int, int, int, int, int, int, int, int) {

int MyVariable1; int MyVariable2; int MyVariable3; int MyVariable4;

int MyVariable5; int MyVariable6; int MyVariable7; int MyVariable8;

int MyVariable9; int MyVariable10; int MyVariable11; int MyVariable12;

int MyVariable13; int MyVariable14; int MyVariable15; int MyVariable16;

return (

MyVariable1, MyVariable2, MyVariable3, MyVariable4, MyVariable5,

MyVariable6, MyVariable7, MyVariable8, MyVariable9, MyVariable10,

MyVariable11, MyVariable12, MyVariable13, MyVariable14, MyVariable15,

MyVariable16);

}

New Error Message

Throw in function void dev::solidity::CompilerUtils::copyToStackTop(unsigned int, unsigned int)

Dynamic exception type: boost::exception_detail::clone_impl<dev::solidity::InternalCompilerError>

std::exception::what: Stack too deep, try removing local variables.

You are probably returning too many parameters from this function.

[dev::tag_comment*] = Stack too deep, try removing local variables.

You are probably returning too many parameters from this function.

=========================================================

There are still some other scenarios which generate this generic error which need to be replicated, however the above scenarios should cover the majority of such occurrences