Getting started with native openGL Android App

During the holidays I wanted to try something different. I never did much mobile development in my life. But since I had a bit of time I thought I ought to give it a go.

But whats the fun of doing something simple, like a form app for example. I wanted to learn something. So I decided to do a simple OpenGL app without using any third party libraries.

Bear in my mind that, once upon a time I was kind of comfortable with OpenGL development (using C++).

I took a look around for tutorials, I found bits and pieces here and there and finally managed to create what I wanted.

Here were my initial requirements

  1. Draw a circle
  2. Make the circle move as you tilt your phone.

Here is what I had in mind

Which was a great opportunity to learn two things

  1. How sensors work
  2. How OpenGL works on Android.

Setting up the project

So I downloaded Android Studio and Created an Empty Project

At this point I’ve read some articles and tutorials so I had some vague Idea what I had to do. I had to create a View and a Renderer

My OpenGLView needs to extend GLSurfaceView

public class OpenGLView extends GLSurfaceView {

public OpenGLView(Context context) {
super(context);
init();
}

public OpenGLView(Context context, AttributeSet attrs) {
super(context, attrs);
init();
}

private void init(){
setEGLContextClientVersion(2); // OpenGL ES Version
setPreserveEGLContextOnPause(true);
setRenderer(new OpenGLRenderer());
}
}

Then my OpenGLRenderer had to implement GLSurfaceView.Renderer

public class OpenGLRenderer implements GLSurfaceView.Renderer {
    @Override
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
GLES20.glClearColor(0.9f, 0.9f,0.9f,1f);
}
   @Override
public void onSurfaceChanged(GL10 gl, int width, int height) {
}
    @Override
public void onDrawFrame(GL10 gl) {
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
}
}

The Renderer at this point doesn’t do anything apart from setting up a background color.

Then its time to go to our layout and add the view we just created and we use that view element to fill our screen.

<net.something.OpenGLPlayground.OpenGLView
android:id="@+id/openGLView"
android:layout_width="match_parent"
android:layout_height="match_parent" />

Finally in our MainActivity (which I named MainGameActivity) should look like this:

public class MainGameActivity extends AppCompatActivity {
private OpenGLView openGLView;
    @Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main_game);
openGLView = findViewById(R.id.openGLView);
}
    @Override
protected void onResume(){
super.onResume();
openGLView.onResume();
}
    @Override
protected void onPause(){
super.onPause();
openGLView.onPause();
}
    @Override
public void onPointerCaptureChanged(boolean hasCapture) {
    }
}

…and if we execute the code. we would get a view with whatever color we defined. The color I defined is nearly white, so there shouldn’t be anything.


Implementing SensorEventListener

Before we draw our circle lets set up the SensorEventListener to listen to the Accelerometer. Ok, the accelerometer might not be the best sensor for what we try to achieve (cause it will only work when you are not moving) so we could switch to the gyroscope but for the time I guess its fine.

public class MainGameActivity extends AppCompatActivity implements SensorEventListener {
...
    @Override
public void onSensorChanged(SensorEvent event) {
if (event.sensor.getType() == Sensor.TYPE_ACCELEROMETER) {
        }
}
    @Override
public void onAccuracyChanged(Sensor sensor, int accuracy) {

}
}

The accelerometer has 3 values

value[0] affects the x axis
value[1] affects the y axis
value[2] affects the z axis

We will use only x and y and we will round the values on the second decimal since we specifically don’t want high accuracy. Our function will look like

@Override
public void onSensorChanged(SensorEvent event) {
if (event.sensor.getType() == Sensor.TYPE_ACCELEROMETER) {
float x = Math.round(event.values[0] * 100.0) / 100f;
float y = Math.round(event.values[1] * 100.0) / 100f;
}
}

Drawing the circle

So we have some coordinates for our circle. So now we need to use the powers of OpenGL to draw the circle.

1. We are going to create a Circle Class with the following functions.

public class Circle {
// basically a circle is a linestring so we need its centre
// radius and how many segments it will consist of
public Circle(float cx, float cy, float radius, int segments) {
}

// calculate the segments
public void calculatePoints(float cx, float cy, float radius, int segments) {
}
    // actuall openGL drawing
public void draw() {
}
}

2. In our Circle Class we are going to add a function that compiles our shape shader code. Basically shaders needs to be compiled by openGL.

public static int loadShader(int type, String shaderCode){
int shader = GLES20.glCreateShader(type);
GLES20.glShaderSource(shader, shaderCode);
GLES20.glCompileShader(shader);
return shader;
}

3. In the Circle Class we will define two shaders (which don’t do much).

private final String vertexShaderCode =
"attribute vec4 vPosition;" +
"void main() {" +
" gl_Position = vPosition;" +
"}";

private final String fragmentShaderCode =
"precision mediump float;" +
"uniform vec4 vColor;" +
"void main() {" +
" gl_FragColor = vColor;" +
"}";

4. Let’s calculate the points of the circle, those points will be stored in a FloatBuffer. Someone will easily spot something weird. That’s the DisplayMetrics on the first line of code of the function. And that’s because OpenGL canvas is square, so the screen coordinates are from -1 to 1 both for x and y. If we just drew the circle it would end up distorted. We need the width and height of the screen to calculate the aspect ratio so we can squeeze one dimension.

private FloatBuffer vertexBuffer;
private static final int COORDS_PER_VERTEX = 3;
public void CalculatePoints(float cx, float cy, float radius, int segments) {
DisplayMetrics dm = Resources.getSystem().getDisplayMetrics();

float[] coordinates = new float[segments * COORDS_PER_VERTEX];

for (int i = 0; i < segments * 3; i += 3) {
float percent = (i / (segments - 1f));
float rad = percent * 2f * (float) Math.PI;

//Vertex position
float xi = cx + radius * (float) Math.cos(rad);
float yi = cy + radius * (float) Math.sin(rad);

coordinates[i] = xi;
coordinates[i + 1] = yi / (((float) dm.heightPixels / (float) dm.widthPixels));
coordinates[i + 2] = 0.0f;
}

// initialise vertex byte buffer for shape coordinates
ByteBuffer bb = ByteBuffer.allocateDirect(coordinates.length * 4);
// use the device hardware's native byte order
bb.order(ByteOrder.nativeOrder());

// create a floating point buffer from the ByteBuffer
vertexBuffer = bb.asFloatBuffer();
// add the coordinates to the FloatBuffer
vertexBuffer.put(coordinates);
// set the buffer to read the first coordinate
vertexBuffer.position(0);
}

5. Time for the constructor. Normally if we just want to draw a shape and be over we don’t need to check if the app has being initialised, but because we intend to update the objects coordinates every time we get a sensor event. We don’t have to load the shader/app/link again and again.

private int app = -1;
public Circle(float cx, float cy, float radius, int segments) {
CalculatePoints(cx, cy, radius, segments);
if (app == -1) {
int vertexShader = OpenGLRenderer.loadShader(GLES20.GL_VERTEX_SHADER,
vertexShaderCode);
int fragmentShader = OpenGLRenderer.loadShader(GLES20.GL_FRAGMENT_SHADER,
fragmentShaderCode);

// create empty OpenGL ES Program
app = GLES20.glCreateProgram();

// add the vertex shader to program
GLES20.glAttachShader(app, vertexShader);

// add the fragment shader to program
GLES20.glAttachShader(app, fragmentShader);

// creates OpenGL ES program executables
GLES20.glLinkProgram(app);
}
}

6. The draw function

public void draw() {

int vertexCount = vertexBuffer.remaining() / COORDS_PER_VERTEX;

// Add program to the environment
GLES20.glUseProgram(app);

// get handle to vertex shader's vPosition member
int mPositionHandle = GLES20.glGetAttribLocation(app, "vPosition");

// Enable a handle to the triangle vertices
GLES20.glEnableVertexAttribArray(mPositionHandle);

// Prepare the triangle coordinate data
GLES20.glVertexAttribPointer(mPositionHandle, COORDS_PER_VERTEX,
GLES20.GL_FLOAT, false,
vertexStride, vertexBuffer);

// get handle to fragment shader's vColor member
mColorHandle = GLES20.glGetUniformLocation(app, "vColor");

// Draw the triangle, using triangle fan is the easiest way
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_FAN, 0, vertexCount);

// Disable vertex array
GLES20.glDisableVertexAttribArray(mPositionHandle);

// Set color of the shape (circle)
GLES20.glUniform4fv(mColorHandle, 1, new float[]{0.5f, 0.3f, 0.1f, 1f}, 0);
}

7. Finally we go back to our Renderer to add a new circle object. We will draw our circle initially at x = 0, y = 0, radius =0.1 and segments 55. objectsReady switch will come in handy when we will start updating the object from the sensor events.

public class OpenGLRenderer implements GLSurfaceView.Renderer {
    private Circle circle;
public boolean objectsReady = false;
public Circle getCircle() {
return circle;
}
    @Override
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
GLES20.glClearColor(0.9f, 0.9f,0.9f,1f);
circle = new Circle(0,0, 0.1f, 55);
objectsReady = true;
}
...
}

At this point if we run our app we should get a brown circle in the middle of our screen. So our onSensorChanged will become. SCALE is just to bring the sensor data range we care about (-4, 4) to our OpenGL view (-1,1).

private final static int SCALE = 4;
@Override
public void onSensorChanged(SensorEvent event) {
if (event.sensor.getType() == Sensor.TYPE_ACCELEROMETER) {
float x = Math.round(event.values[0] * 100.0) / 100f;
float y = Math.round(event.values[1] * 100.0) / 100f;
if (openGLView.renderer.objectsReady) {
openGLView.renderer.getCircle().CalculatePoints(x / SCALE, y / SCALE, 0.1f, 55);
openGLView.requestRender();
}

}
}

Finally our circle, is alive and well, but its motion is jitterish.


Normalise sensor data

SensorEventListener fires thousand events per second with a huge accuracy, so in order to make movement smooth, we need to normalise our data by using some statistical method. The obvious choice with this problem (at least obvious to me) is to use moving average.

The moving average is nothing more than the average value of the x last readings.

This is easily done. We only need to add the following to our MainActivity.

private final static int OVERFLOW_LIMIT = 20;
private float[][] movingAverage = new float[2][OVERFLOW_LIMIT];
@Override
public void onSensorChanged(SensorEvent event) {
if (event.sensor.getType() == Sensor.TYPE_ACCELEROMETER) {
float x = Math.round(event.values[0] * 100.0) / 100f;
float y = Math.round(event.values[1] * 100.0) / 100f;

movingAverage[0][overflow] = x;
movingAverage[1][overflow] = y;

float s1 = calculateAverage(movingAverage[0]);
float s2 = calculateAverage(movingAverage[1]);


if (openGLView.renderer.objectsReady) {
openGLView.renderer.getCircle().CalculatePoints(s1 / SCALE, s2 / SCALE, 0.1f, 55);
openGLView.requestRender();
}
}
overflow += 1;
if (overflow >= OVERFLOW_LIMIT) {
overflow = 0;
}
}
private float calculateAverage(float[] input) {
DoubleStream io = IntStream.range(0, input.length)
.mapToDouble(i -> input[i]);
float sum = (float)io.sum();
return sum/OVERFLOW_LIMIT;
}

Running the app again, we now get a smoother motion of our shape.

This article, took me 4 days and ended up quite big. There is a high chance that I have a few mistakes here and there. If you notice anything, let me know and I’ll amend it.