APEX Behind the Scenes #2 — Guest Post by CTO Richard Wang — Part 1
I know many who have been following Apex Network for some time now, have been wanting to know more about the core team; who we are and what we do. Jimmy is our main public figure as CEO, especially towards the token holders, but we felt like it was time to introduce ourselves and let you know some of our thoughts. That is why I am picking up where Jimmy left off last time, with our Behind the Scenes — Part II. I intend to provide some insights into who I am, what I do, how the development is going, and write out some of my thoughts about the project — in part based on feedback I have received about questions you guys in the community have, and in part based on information that I feel is relevant to you guys as investors in the project. So without further ado, we’ll dive right into it.
Who am I in Apex Network, and what do I really do?
Firstly, it’s important to know what exactly my role as CTO of Apex Network really entails. My responsibilities can be summed up as follows;
1) Setting and designing the short and long term technology strategy for the project — this includes architecture, feasibility, implementation methodologies, and technical-level innovation/ experimentation.
2) Creating a high quality talent strategy for the tech organization supporting the project, which includes recruitment strategy, training development, and talent incentivization and retention.
3) Making sure development is delivered in a timely manner while ensuring a level quality and robustness.
My workday starts at 9:30 am. On normal days without any special arrangements or meetings planned, the blockchain development team has a progress meeting at 11:00 am. This serves as a daily overview session for briefing and task management. The progress meetings are split into two groups — the blockchain platform group and the applications group (which mainly focuses on the APEX Wallet) — and these meetings will include members of the project management team as well. In the afternoons, there are often project-specific meetings or partner meetings (i.e. AWS, Microsoft, enterprise customers, blockchain projects, etc) that may involve the engineering team.
In addition to this, we have a hands-on session focused on testing and resolving specific tech issues 2–3 times per week. These sessions are usually led by me or Rui, one of our head architects for the blockchain division. I personally meet with the management of APEX Network 3–4 times a week, including Jimmy to discuss progress, strategy, and talent management.
I spend most of my evenings replying to emails, preparing meetings for the next day, reading up on tech news and playing sports.
As I am CTO of Apex Technologies as well as Apex Network, some have probably wondered how I manage and divide my time between the two. Firstly, 90% of my time is focused on management, and I would say the remaining 10% I get involved in crunching out some code myself when necessary. Of the time spent on management, roughly 40% of the time is spent on APEX Network and 60% of the time on Apex Technologies. This signifies quite a bit of management put into the blockchain counterpart, as the Apex Technologies engineering team is significantly larger and running much more complex operations as well.
Staying current on the latest in blockchain tech developments
For me personally, one of the most effective ways to keep my knowledge up to date is through regular casual conversations with blockchain technology teams and leaders whom I bump into quite often, especially in the Greater China region and especially at conferences and meetups (I was just at a large one in Shanghai this week). We share insights on development progress and tentative tech plans, albeit technology used and design of the platform could be quite different project-to-project.
My engineering team feeds me quite a bit of information on a regular basis as well, as they are involved in day-to-day technical work. We also on a regular basis communicate with respective teams of large organizations using blockchain technology or involved in blockchain infrastructure — this may include infrastructure providers such as Azure, AWS, Oracle, Alicloud, as well as B2C enterprises experimenting with blockchain tech (airlines, automotive companies, etc).
In the very little spare time I may have, sometimes I check out the codebase and technical design of the newest infrastructure projects, as well as of projects that have been around for some time, my favorites of which include Lisk, Ontology, and Vechain.
One should not underestimate the importance of learning from other blockchain projects around us. When we’re open to learning from our peers, we benefit from their experience as well as our own and we can inherit their knowledge and wisdom.
The team I work with
I know a lot of you have been asking about our team; how many people are on it, what their tasks are, how many are working on a specific area of development etc. So let’s talk about my team a bit.
Currently 15 engineers are working in the blockchain division (out of 70 engineers total in the company, Apex Technologies), with 3–4 focused on applications at this moment. There are 6 project management team members in the blockchain division, bringing the entire team at this current time to 21.
The lead engineers on the blockchain team mostly come from Alibaba and Tencent, and prior to joining our company they worked in their server infrastructure teams. We also have a couple from enterprise software companies such as EMC, Redhat, and Huawei. Two of the engineers worked at the private blockchain teams of fintech companies (such as Pingan Technology).
Around half of the engineers on the blockchain team were transferred internally from Apex Technologies to the blockchain division, and the other half are new recruits. Now, our hiring strategy is quite different from typical blockchain projects — we focus on raw foundation (understanding of engineering, systems, architecture), intelligence (learning ability, problem solving ability), and track record at scale (building scalable systems at reputable technology companies) rather than “blockchain experience”, which we in most cases find to be the least valuable.
I won’t go into detail about the rationale for this, but the gist of it is :
- The market and technology is at a rather early stage, to the extent that if a candidate claims to have “experience” it probably won’t mean much moving forward. In some cases, some early knowledge and preconceptions may not help developing a new infrastructure project using a different approach and technology than the candidate was previously accustomed to.
- Especially in China, most candidates on the market looking for blockchain engineering jobs and claiming to have “blockchain experience” are in the low-mid to mid-tier in terms of caliber compared to those from top technology companies. This is a phenomenon that’s relatively easy to explain:
A) The top engineers already have good pay grade jobs at top technology firms and venture-backed AI/big data/fintech companies — they have no need to go into a riskier field, blockchain, with its crypto-associated stigma.
B) The typically less qualified engineers, with a bit of self-learned blockchain concepts and knowledge of Solidity, can command much higher salaries than their regular pay grade if they go into blockchain.
- Engineers with a solid “raw foundation” as described above will be able to pick up blockchain infrastructure development quite easily. Their past experience creating deep-learning models and programming database kernels or telecom protocols may be more challenging work than blockchain engineering.
Ok, that’s enough about who we are — time to talk a little about what we do. What’s going on with the wallet, and particularly with the blockchain we’re developing?
So what’s next for the Apex wallet?
Firstly we’ll be integrating ERC-20 functionality quite soon. After ERC-20 gets implemented, for the wallet we will focus on a couple things:
- Multi-account and cross-blockchain account management features
- Features related to the enterprise ecosystem and user consumption (consumer users)
- In general features and levers that will grow the user base, awareness, and community.
Why did we choose our own blockchain vs building on top of Neo, Ethereum, EOS or another existing blockchain?
I think we’ve justified this question quite a few times in the past. It’s really a needs-driven decision that we came to after receiving feedback from the enterprise users (in large part initially our existing customers) who are the ultimate adopters. They’re not comfortable with building on a public blockchain like Neo, Ethereum or EOS — this is something that most people do not see. Back in Q4 2017, we presented tentative solutions built on NEO and Ethereum to enterprise customers that were willing to experiment with blockchain — they saw no need to build applications on a public blockchain not catered to the use case at hand. In other words, to them it’s not a real “solution” and not “close to delivering value”. On the other hand, a company wanting to build blockchain applications surrounding supply chain use cases will probably find a platform like Vechain helpful.
We’ll see more blockchain infrastructure projects spin out of organizations and teams that are experienced with enterprise software and who have a sizable enterprise customer base. These will be the types of teams that can really start to ease adoption in real-world use cases and provide a real “solution” that enterprises can quickly see value in.
Why did we choose a mix of Scala and Go for the blockchain and what are the advantages of using these languages?
A major part of the blockchain will be written in Scala, which is a rather uncommon choice for blockchain projects. The reasoning is as follows:
- Scala is the most widely used language within Apex Technologies as a company. Simply put, we are experts in the language and have access to talent within the larger engineering team (70+ people) when the blockchain team needs more engineering resources. We are also able to train new recruits easily.
- Scala is a highly versatile language for enterprise applications surrounding server development, data, artificial intelligence, and middleware. It runs on the highly robust, fast performing, and enterprise-ready JVM (Java Virtual Machine). The runtime speed for certain applications is faster than Java, and much faster than Python and Erlang.
- It’s a functional programming language that, once mastered, can deliver the same with much less lines of code than languages like Java and C++, implying better productivity.
- Scala integrates well with Java and Java libraries, which means that once a larger engineering team is needed and when Scala engineers become scarcer, part of the codebase can use Java as well.
How is the Testnet development progressing? What are we currently working on and how does GitHub fit into this? …and what about that “code review”?
For the blockchain, we are internally testing core modules, including transactions, block production, and consensus. In other words Testnet is on track. Blockchain explorer app development has started, and like I mentioned earlier we are releasing a new version of the Apex Wallet with ERC20 support very soon.
I’ve heard a lot of talk about GitHub, the amount of code published etc, so it seems prudent to clarify a few things in that regard. First of all; we pretty much never use Github internally to manage our repositories. It was created purely for the purpose of putting some code out there for the community to see. There’s only one person from each team (app + blockchain) intermittently uploading certain parts of code on to Github when he gets around to it. Typically, at any given time, it’s around 80% of the entire codebase that we have.
If anyone is wondering — yes, I know about the CryptoBriefing post. I am honestly not sure it really warrants a response, but I hear it’s been a subject of interest in the community after it was published. I did head over to read the article — it’s actually a bit funny. Before I say anything about the contents, a couple of things strike me as quite interesting:
- The tone of the article is a bit emotionally charged
- The author’s credentials would only qualify as an entry-level engineer at Apex Technologies
- The review is far from comprehensive — the author only chose code snippets which he can seemingly negate.
Now, in the article the author mentioned something along the lines of “copied Bitcoin, but in Scala”. First, Bitcoin is not written in Scala, and did you mean UTXO, a transaction model used by several prominent blockchains other than Bitcoin? And on the consensus — our mistake, the crux of the code had not been updated at the time of the review.
Nevertheless, anyone is welcome to have their own opinion — I would welcome critiques with open arms once Testnet is up and running. For those who are truly interested in reviewing the code at some point, I can confirm that the plan is to make all the code open source and publish it on GitHub after Mainnet has launched. I trust that you will be happy with what you see.
That’s it for this post guys. There are lots of things the team is working on these days, really quite exciting times. I have some more thoughts I would like to share with you guys and hope to be able to publish these in a week from now. And yes, we will touch on the nodes! This post is already running quite long as it stands. As Jimmy would say; “stay tuned!”
CTO APEX Network, CTO APEX Technologies