Extracting Data using TypeScript Compiler API- Week- 2 GSoC’21
Hey Welcome to week 2 of GSoC where I would be discussing about work progress and will also understand how data can be extracted from a code using TypeScript compiler API
How to traverse nodes in AST
Before starting on what I am doing lets understand how to visit nodes in an Abstract Syntax tree, in best possible ways. We know one method which is ast.ForEachChild() but if we want to go in depth of each child then other methods are need to be used.
When wanting to modify the AST in any way you need to traverse the tree — recursively. In more concrete terms we want to visit each node, and then return either the same, an updated, or a completely new node.
Lets Take this source code
{
kind: 288, // (SyntaxKind.SourceFile)
statements: [{
kind: 243, // (SyntaxKind.FunctionDeclaration)
name: {
kind: 75 // (SyntaxKind.Identifier)
escapedText: "hello"
},
body: {
kind: 222, // (SyntaxKind.Block)
statements: [{
kind: 225, // (SyntaxKind.ExpressionStatement)
expression: {
kind: 195, // (SyntaxKind.CallExpression)
expression: {
kind: 193, // (SyntaxKind.PropertyAccessExpression)
name: {
kind: 75 // (SyntaxKind.Identifier)
escapedText: "log",
},
expression: {
kind: 75, // (SyntaxKind.Identifier)
escapedText: "console",
}
}
},
arguments: [{
kind: 10, // (SyntaxKind.StringLiteral)
text: "world",
}]
}]
}
}]
}visitNode()
This is one of the function in typescript module to visit nodes
import * as ts from 'typescript';
ts.visitNode(sourceFile, visitor);visitEachChild()
Visit child uses visit node internally, visiting each child of node in depth
import * as ts from 'typescript';
ts.visitEachChild(node, visitor, context);visitor
If combining the above two and the visitor pattern, here is the code
const transformer = sourceFile => {
const visitor = (node: ts.Node): ts.Node =>
console.log(node.kind,`\t#ts.SyntaxKind.${ts.SyntaxKind[node.kind]}` );
return ts.visitEachChild(node, visitor, context);
};
return ts.visitNode(sourceFile, visitor);
};The output would be
288 # ts.SyntaxKind.SourceFile
243 # ts.SyntaxKind.FunctionDeclaration
75 # ts.SyntaxKind.Identifier
222 # ts.SyntaxKind.Block
225 # ts.SyntaxKind.ExpressionStatement
195 # ts.SyntaxKind.CallExpression
193 # ts.SyntaxKind.PropertyAccessExpression
75 # ts.SyntaxKind.Identifier
75 # ts.SyntaxKind.Identifier
10 # ts.SyntaxKind.StringLiteralIt goes as deep as possible entering each node, exiting when it bottoms out, and then entering other child nodes that it comes to.
What does Extracting Data mean
So as you know we have already cloned a class node while parsing, basically it is a class you write in javascript but in an ast format. Now the work is to get functions, there parameters and other important details of how class is built so that we can work upon it and transform or change it according to our needs
Lets first Understand our cloned class
We have class->members->Method Declaration->Parameters
So basically class contains members, members can be functions, constructors and return statements, we are bothered about only method declaration node, method declaration node(Function node) contains a lot of stuff but we are mostly bothered about parameters it contains along with its own name.
So we will be extracting these functions, there parameters and name to pass it for transforming.
These are the interface of the data that is to be extracted
interface FunctionData {
functionName: string;
SDKFunctionName: string;
params: param[];
}interface param {
name: string;
type: string;
typeName: string;
}interface ClassData {
className: string;
functions: FunctionData[];
serviceName: string;
}
So above are the interface which defines how data looks like with there types
How is Data Extraction Done
As we have the class ast with us, we will start visiting all its child nodes and if we encounter a syntax kind of Method Declaration, we will push it and then we will again traverse through the extracted function nodes to extract there child parameter nodes.
sdkClassAst.members.map(method => { if (method.name && functions.includes(method.name.text)) {
// Do stuff with the function
}
})
Inside this ‘If’ I will also extract Parameters of this function, store parameters in an array and also the extracted functions in another array
let methods: FunctionData[] = [];sdkClassAst.members.map(method => { if (method.name && functions.includes(method.name.text)) {
const parameters = [];
method.parameters.map(param => {
if (param.name.text !== "callback") {
const parameter = {
name: param.name.text,
optional: param.questionToken ? true : false,
type: SyntaxKind[param.type.kind],
typeName: null
}; if (parameter.type==="TypeReference" && param.type.typeName) {
parameter.typeName = param.type.typeName.text;
} parameters.push(parameter);
}
}); methods.push({
functionName: name.toString(),
SDKFunctionName: method.name.text.toString(),
params: parameters
});});
At last Storing all functions in a class data object with class name and then returning it
const classData: ClassData = {
className: sdkClassAst.name.text,
functions: methods,
serviceName: null
};return classData;
So in this way, we extracted all the data that was needed of a class and will be doing some transformation on a dummy Class and merging this data in that class.
My Weekly update
So I did all the above stuff of passing the AST made using parser written previously and then using that ast visiting all the nodes and child nodes and extracting data from it, like Class name, Functions name, there parameters and the functions name. Storing it in objects of array and setting it ready to be sent to transformer for transformation and merging with a dummy class.
This week was also a great fun, where I brainstormed on few points with my mentor. I also met my another mentor Rumesh Hapuarachchi, discussing with him the progress made till now.
Now In my next week, I will be working on transformation part.
So Till then GoodBye✌️ and Have Fun!😋
