How I test with Apache Spark?

Nastasia Saby
Jul 30 · 3 min read
Spark photo by @jeztimms on Unsplash

What to test with Apache Spark?

Spark Session in our tests

Spark Session in a wrapper

External files VS DataFrames/Datasets creation

Performance

Run once globally, test unitarly

Conclusion

Nastasia Saby

Written by

#SoftwareCrafting and #BigData fan @Zenika

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade