The Klinify Blog
Published in

The Klinify Blog

How Pyroscope Saved Us Weeks of Wasted Effort

A Lesson in Continuous Profiling

Photo by Ilona Frey on Unsplash

Disclaimer: Please don’t slow down when a snake jumps at you from behind a bush.

from werkzeug.middleware.profiler 
import ProfilerMiddleware
app = ProfilerMiddleware(app)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pyroscope-claim0
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: default
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: pyroscope
name: pyroscope
spec:
replicas: 1
selector:
matchLabels:
app: pyroscope
strategy:
type: Recreate
template:
metadata:
labels:
app: pyroscope
spec:
containers:
- name: pyroscope
image: pyroscope/pyroscope:0.0.29
command: ["pyroscope"]
args: ["server"]
ports:
- containerPort: 4040
resources:
limits:
memory: 500Mi
requests:
memory: 150Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/lib/pyroscope
name: pyroscope-storage
initContainers:
- name: pyroscope-pv-permission-fix
image: busybox
command: ["chmod", "-R", "777", "/data"]
volumeMounts:
- name: pyroscope-storage
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
volumes:
- name: pyroscope-storage
persistentVolumeClaim:
claimName: pyroscope-claim0
status: {}
apiVersion: v1
kind: Service
metadata:
labels:
app: pyroscope
name: pyroscope
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
ports:
- port: 4040
targetPort: 4040
selector:
app: pyroscope
type: LoadBalancer

Note the internal: true for the LB. Pyroscope did not have a login system back when I implemented this, so I put it behind the Devs' VPN. They’ve implemented auth recently, check it out at https://pyroscope.io/docs/auth-google

...
COPY --from=pyroscope/pyroscope:latest /usr/bin/pyroscope /usr/bin/pyroscope
ENV PYROSCOPE_SPY_NAME=pyspy
ENV PYROSCOPE_SERVER_ADDRESS=http://pyroscope:4040
ENV PYROSCOPE_APPLICATION_NAME=klinify.api
...
CMD ["/start.sh"]
securityContext:
capabilities:
add:
- SYS_PTRACE
...
exec pyroscope exec uwsgi --ini /app/uwsgi.ini

Note: I believe Pyroscope now has integrations for different languages that you can call within the code. They’ve also released Helm charts to make setting the Server up easier. Look at that if you want to try out Pyroscope!

Note: This is not to say Pyroscope is perfect, there are missing some features (like any other project). If you want something, submit an issue or make a PR on Github. The authors are very friendly and you should join their Slack community if you have questions 🙂

Story originally published at: https://medium.com/dabbler-in-de-stress/how-pyroscope-saved-me-weeks-of-wasted-effort-dc99aeaf29e9

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store