Search Results for: spark
-
Enterprise Java
Apache Spark Packages, from XML to JSON
The Apache Spark community has put a lot of effort into extending Spark. Recently, we wanted to transform an XML…
Read More » -
Software Development
Tutorial: Using PySpark and the MapR Sandbox
PySpark is a Spark API that allows you to interact with Spark through the Python shell. If you have a…
Read More » -
DevOps
Testing Spark Streaming: Integration testing with Docker Compose
In the first post of this series, we saw how to unit test Spark Streaming operations using Spark Testing Base.…
Read More » -
Software Development
Testing Spark Streaming: Unit testing
There is enough evidence to prove the importance of automated testing. Projects in new fields often neglect automated testing, as…
Read More » -
Software Development
How to Speed Up Ad-hoc Analytics with SparkSQL, Parquet, and Alluxio
In the big data enterprise ecosystem, there are always new choices when it comes to analytics and data science. Apache…
Read More » -
Software Development
Persistent Storage for Enterprise-Grade Spark Applications
Apache Spark is becoming very popular and widely used in the big data community. There are several reasons for Spark…
Read More » -
Software Development
A Functional Approach to Logging in Apache Spark
Logging in Apache Spark is very easy to do, since Spark offers access to a logobject out of the box;…
Read More » -
Software Development
From Pig to Spark: An Easy Journey to Spark for Apache Pig Developers
As a data analyst that primarily used Apache Pig in the past, I eventually needed to program more challenging jobs…
Read More » -
Software Development
Using Apache Spark SQL to Explore S&P 500, and Oil Stock Prices
This post will use Apache Spark SQL and DataFrames to query, compare and explore S&P 500, Exxon and Anadarko Petroleum…
Read More »