Tweeting StackExchange with Spring Social – part 1

This article will cover a quick side-project – a bot to automatically tweet Top Questions from the various Q&A StackExchange sites, such as StackOverflow, ServerFault, SuperUser, etc. We will build a simple client for the StackExchange API and then we’ll setup the interaction with the Twitter API using Spring Social – this first part will focus on the StackExchange Client only. The initial purpose of this implementation is not to be a full fledged Client for the entire StackExchange API – that would be outside the scope of this project. The only reason the Client exists is that I couldn’t fine one that would work against the 2.x version of the official API.
 
 
 

1. The Maven dependencies

To consume the StackExchange REST API, we will need very few dependencies – essentially just an HTTP client – the Apache HttpClient will do just fine for this purpose:

<dependency>
   <groupId>org.apache.httpcomponents</groupId>
   <artifactId>httpclient</artifactId>
   <version>4.2.3</version>
</dependency

The Spring RestTemplate could also have been used to interact with the HTTP API, but that would have introduced quite a lot of other Spring related dependencies into the project, dependencies which are not strictly necessary, so HttpClient will keep things light and simple.

2. The Questions Client

The goal of this Client is to consume the /questions REST Service that StackExchange publishes, not to provide a general purpose client for the entire StackExchange APIs – so for the purpose of this article we will only look at that. The actual HTTP communication using HTTPClient is relatively straightforward:

public String questions(int min, String questionsUri) {
   HttpGet request = null;
   try {
      request = new HttpGet(questionsUri);
      HttpResponse httpResponse = client.execute(request);
      InputStream entityContentStream = httpResponse.getEntity().getContent();
      return IOUtils.toString(entityContentStream, Charset.forName('utf-8'));
   } catch (IOException ex) {
      throw new IllegalStateException(ex);
   } finally {
      if (request != null) {
         request.releaseConnection();
      }
   }
}

This simple interaction is perfectly adequate for obtaining the questions raw JSON that the API publishes – the next step will be processing that JSON. There is one relevant detail here – and that is the questionsUri method argument – there are multiple StackExchange APIs that can publish questions (as the official documentation suggests), and this method needs to be flexible enough to consume all of them. It can consume for example the simplest API that returns questions by setting questionUri set to https://api.stackexchange.com/2.1/questions?site=stackoverflow or it may consume the tag based https://api.stackexchange.com/2.1/tags/{tags}/faq?site=stackoverflow API instead, depending on what the client needs.

A request to the StackExchange API is fully configured with query parameters, even for the more complex advanced search queries – there is no body being send. To construct the questionsUri, we’ll build a basic fluent RequestBuilder class that will make use of the URIBuilder from the HttpClient library. This will take care of correctly encoding the URI and generally making sure that the end result is valid:

public class RequestBuilder {
   private Map<String, Object> parameters = new HashMap<>();
   public RequestBuilder add(String paramName, Object paramValue) {
       this.parameters.put(paramName, paramValue);
      return this;
   }
   public String build() {
      URIBuilder uriBuilder = new URIBuilder();
      for (Entry<String, Object> param : this.parameters.entrySet()) {
         uriBuilder.addParameter(param.getKey(), param.getValue().toString());
      }
      return uriBuilder.toString();
   }
}

So now, to construct a valid URI for the StackExchange API:

String params = new RequestBuilder().
   add('order', 'desc').add('sort', 'votes').add('min', min).add('site', site).build();
return 'https://api.stackexchange.com/2.1/questions' + params;

3. Testing the Client

The Client will output raw JSON, but to test that, we will need a JSON processing library, specifically Jackson 2:

<dependency>
   <groupId>com.fasterxml.jackson.core</groupId>
   <artifactId>jackson-databind</artifactId>
   <version>2.1.3</version>
   <scope>test</scope>
</dependency>

The tests we’ll look at will interact with the actual StackExchange API:

@Test
public void whenRequestIsPerformed_thenSuccess()
      throws ClientProtocolException, IOException {
   HttpResponse response = questionsApi.questionsAsResponse(50, Site.serverfault);
   assertThat(response.getStatusLine().getStatusCode(), equalTo(200));
}
@Test
public void whenRequestIsPerformed_thenOutputIsJson()
      throws ClientProtocolException, IOException {
   HttpResponse response = questionsApi.questionsAsResponse(50, Site.serverfault);
   String contentType = httpResponse.getHeaders(HttpHeaders.CONTENT_TYPE)[0].getValue();
   assertThat(contentType, containsString('application/json'));
}
@Test
public void whenParsingOutputFromQuestionsApi_thenOutputContainsSomeQuestions()
     throws ClientProtocolException, IOException {
   String questionsAsJson = questionsApi.questions(50, Site.serverfault);
   JsonNode rootNode = new ObjectMapper().readTree(questionsAsJson);
   ArrayNode questionsArray = (ArrayNode) rootNode.get('items');
   assertThat(questionsArray.size(), greaterThan(20));
}

The first test has verified that the response provided by the API was indeed a 200 OK, so the GET request to retrieve the questions was actually successful. After that basic condition is ensured, we moved on to the Representation – as specified by the Content-Type HTTP header – that needs to be JSON. Next, we actually parse the JSON and verify that there are actually Questions in that output – that parsing logic itself is low level and simple, which is enough for the purpose of the test. Note that these requests count towards your rate limits specified by the API – for that reason, the Live tests are excluded from the standard Maven build:

<plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-surefire-plugin</artifactId>
   <version>2.13</version>
   <configuration>
      <excludes>
         <exclude>**/*LiveTest.java</exclude>
      </excludes>
   </configuration>
</plugin>

4. The Next Step

The current Client is only focused on a single type of Resource from the many available types published by the StackExchange APIs. This is because it’s initial purpose is limited – it only needs to enable a user to consume Questions from the various sites in the StackExchange portfolio. Consequently, the Client can be improved beyond the scope of this initial usecase to be able to consume the other types of the API. The implementation is also very much raw – after consuming the Questions REST Service, it simply returns JSON output as a String – not any kind of Questions model out of that output. So, a potential next step would be to unmarshall this JSON into a proper domain DTO and return that back instead of raw JSON.

5. Conclusion

The purpose of this article was to show how to start building an integration with the StackExchange API, or really an HTTP based API out there. It covered how to write integration tests against the live API and make sure the end to end interaction actually works.

The second part of this article will show how to interact with the Twitter API by using the Spring Social library, and how to use the StackExchange Client we built here to tweet questions on a new twitter account.

I have already set up a few twitter accounts that are now tweeting the 2 Top Questions per day, for various disciplines:

  • SpringAtSO – Two of the best Spring questions from StackOverflow each day
  • JavaTopSO – Two of the best Java questions from StackOverflow each day
  • AskUbuntuBest – Two of the best questions from AskUbuntu each day
  • BestBash – Two of the best Bash questions from all StackExchange sites each day
  • ServerFaultBest – Two of the best questions from ServerFault each day

The full implementation of this StackExchange Client is on github.
 

Reference: Tweeting StackExchange with Spring Social – part 1 from our JCG partner Eugen Paraschiv at the baeldung blog.

Related Whitepaper:

Functional Programming in Java: Harnessing the Power of Java 8 Lambda Expressions

Get ready to program in a whole new way!

Functional Programming in Java will help you quickly get on top of the new, essential Java 8 language features and the functional style that will change and improve your code. This short, targeted book will help you make the paradigm shift from the old imperative way to a less error-prone, more elegant, and concise coding style that’s also a breeze to parallelize. You’ll explore the syntax and semantics of lambda expressions, method and constructor references, and functional interfaces. You’ll design and write applications better using the new standards in Java 8 and the JDK.

Get it Now!  

Leave a Reply


8 × five =



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below:
Close