PySpark and Scala Spark

GatewayServer gs = new GatewayServer(new StackEntryPoint());
gs.start();
from py4j.java_gateway import JavaGateway
gw = JavaGateway() # connect to the JVM
java_object = gw.jvm.mypackage.MyJavaClass() # invoke constructor
other_object = java_object.doThat()
gateway.jvm.java.lang.System.out.println('Hello World!')
Author: Piotr Łusakowski, https://deepsense.ai/cooperative-data-exploration/

References

  1. Cooperative data exploration
  2. How does PySpark work? — step by step (with pictures)
  3. https://stackoverflow.com/a/3793523

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store