Featured FREE Whitepapers

What's New Here?

software-development-2-logo

Essential Attack Surface Management

To attack your system, to steal something or do something else nasty, the bad guys need to find a way in, and usually a way out as well. This is what Attack Surface Analysis is all about: mapping the ways in and out of your system, looking at the system from an attacker’s perspective, understanding what parts of the system are most vulnerable, where you need to focus testing and reviews. It’s part of design and it’s also part of risk management. Attack Surface Analysis is simple in concept. It’s like walking around your house, counting all of the doors and windows, and checking to see if they are open, or easy to force open. The fewer doors and windows you have, and the harder they are to open, the safer you are. The bigger a system’s attack surface, the bigger a security problem you have, and the more work that you have to put into your security program. For enterprise systems and web apps, the doors and windows include web URLs (every form, input field – including hidden fields, URL parameters and scripts), cookies, files and databases shared outside the app, open ports and sockets, external system calls and application APIs, admin user ids and functions. And any support backdoors into the app, if you allow that kind of thing. I’m not going to deal with minimizing the attack surface by turning off features or deleting code. It’s important to do this when you can of course, but most developers are paid to add new features and write more forms and other interfaces – to open up the Attack Surface. So it’s important to understand what this means in terms of security risk. Measuring the System’s Attack Surface Michael Howard at Microsoft and other researchers have developed a method for measuring the attack surface of an application, and to track changes to the attack surface over time, called the Relative Attack Surface Quotient (RSQ). Using this method you calculate an overall attack surface score for the system, and measure this score as changes are made to the system and to how it is deployed. Researchers at Carnegie Mellon built on this work to develop a formal way to calculate an Attack Surface Metric for large systems like SAP. They calculate the Attack Surface as the sum of all entry and exit points, channels (the different ways that clients or external systems connect to the system, including TCP/UDP ports, RPC end points, named pipes…) and untrusted data elements. Then they apply a damage potential/effort ratio to these Attack Surface elements to identify high-risk areas. Smaller teams building and maintaining smaller systems (which is most of us) and Agile teams trying to move fast don’t need to go this far. Managing a system’s attack surface can be done through a few straightforward steps that developers can understand and take ownership of. Attack Surface: Where to Start? Start with some kind of baseline if you can – at least a basic understanding of the system’s attack surface. Spend a few hours reviewing design and architecture documents from an attack surface perspective. For web apps you can use a tool like Arachni or Skipfish or w3af or one of the many commercial dynamic testing and vulnerability scanning tools or services to crawl your app and map the attack surface – at least the part of the system that is accessible over the web. Or better, get an appsec expert to review the application and pen test it so that you understand the attack surface and real vulnerabilities. Once you have a map of the attack surface, identify the high risk areas. Focus on remote entry points – interfaces with outside systems and to the Internet – and especially where the system allows anonymous, public access. This is where you are most exposed to attack. Then understand what compensating controls you have in place, operational controls like network firewalls and application firewalls,and intrusion detection or prevention systems to help protect your app. The attack surface model will be rough and incomplete to start, especially if you haven’t done any security work on the system before. Use what you have and fill in the holes as the team makes changes to the attack surface. But how do you know when you are changing the attack surface? When are you Changing the Attack Surface? According to The Official Guide to the CSSLP “… it is important to understand that the moment a single line of code is written, the attack surface has increased.” But this over states the risk of making code changes – there are lots of code changes (for example to behind-the-scenes reporting and analytics and changes to business logic) that don’t make the system more vulnerable to attack. Remember, the attack surface is the sum of entry and exit points and untrusted data elements in the system. Adding a new system interface, a new channel into the system, a new connection type, a new API, a new type of client, a new mobile app, or a new file or database table shared with the outside – these changes directly affect the Attack Surface and change the risk profile of your app. The first web page that you create opens up the system’s attack surface significantly and introduces all kinds of new risks. If you add another field to that page, or another web page like it, while technically you have made the attack surface bigger, you haven’t increased the risk profile of the application in a meaningful way. Each of these incremental changes is more of the same, unless you follow a new design or use a new framework. Changes to session management and authentication and password management also affect the attack surface. So do changes to authorization and access control logic, especially adding or changing role definitions, adding admin users or admin functions with high privileges. Changes to the code that handles encryption and secrets. Changes to how data validation is done. And major architectural changes to layering and trust relationships, or fundamental changes in technical architecture – swapping out your web server or database platform, or changing the run-time OS. Use a Risk-Based and Opportunity-Based Approach Attack Surface management can be done in an opportunistic way, driven by your ongoing development requirements. As they work on a piece of the system, the team reviews whether and how the changes affect the attack surface, what the risks are, and raise flags for deeper review. These red flags drive threat modeling and secure code reviews and additional testing. This means that developers can stay focused on delivering features, while still taking responsibility for security. Attack surface reviews become a part of design and QA and risk management, burned in to how the team works, done when needed in each stage or phase or sprint. The first time that you touch a piece of the system it may take longer to finish the change because you need to go through more risk assessment. But over time, as you work on the same parts of the system or the same problems, and as you learn more about the application and more about security risks, it will get simpler and faster. Your understanding of the system’s attack surface will probably never be perfect or complete – but you will always be updating it and improving it. New risks and vulnerabilities will keep coming up. When this happens you add new items to your risk checklists, new red flags to watch for. As long as the system is being maintained, you’re never finished doing attack surface management. Reference: Essential Attack Surface Management from our JCG partner Jim Bird at the Building Real Software blog....
jcg-logo

Best Of The Week – 2012 – W06

Hello guys, Time for the “Best Of The Week” links for the week that just passed. Here are some links that drew Java Code Geeks attention: * You need a code review: This article praises the merits of code reviews and discusses its benefits such as creating code quality, preventing bugs and mistakes, transferring knowledge, teaching less experienced developers and more. Also check out Code reviews in the 21st Century and Not doing Code Reviews? What’s your excuse?. * A business component architecture with Spring 3.0/3.1 – Part 3: Properties: In this tutorial, handling configuration properties with Spring is examined. Reading properties is infrastructure code and therefore should ne separated from business logic. Also check out Properties with Spring. * Identity Management with Spring Security: In this presentation, identity management with Spring security is discussed along with SSO, security standards – SAML, OpenID, OAuth, SCIM, JWT-, how Spring Security fits in, and demoing IdM as a service. Also check out OAuth with Spring Security and Single Sign On for the cloud: SAML & OpenId. * UI Construction with JavaFX: This tutorial presents some principles on UI construction and describes how to leverage them with JavaFX for creating widgets, applying the correct layout and style, performing data binding and attaching handlers. * Are You a Zen Coder or Distraction-Junkie?: Must read article on programmer’s productivity. Examines the differences between Zen coder (clean cut, focused) and the distraction junkie (mind all over the place). As stated, the key to true productivity and efficiency is to focus 100% on the one thing you are doing at the moment, and then to completely switch and do something else. Good stuff… * Jdbc Logging with a Proxy Driver: jdbcdslog: Quick tutorial on how to use jdbcdslog (a JDBC proxy driver) for enabling JDBC logging. This approach can help you to debug tricky DB related issues. * Get in control with Spring Insight!: This tutorial presents how to writing your own Spring Insight plugin. With Spring Insight it is possible to gather performance information about your web application and it offers a default set of plugins/inspections for different kinds of frameworks/libraries like Spring Core and Web, JDBC, Servlets, Hibernate and Grails. Also check out Spring Insight – Web Application Profiling. * Performance Analysis of REST/HTTP Services with JMeter and Yourkit: Nice article on how to combine JMeter and Yourkit for perfomance analysis and profiling of REST and HTTP based services. This combination allows us to troubleshoot performance and scaling issues. * Java EE on Google App Engine: CDI to the Rescue: In this presentation, several tips and tricks are shared on how to use top Java EE specs, CDI, JPA, JSF2, and Bean Validation within GAE’s restrictive sandbox while still benefiting from the highly scalable environment it provides and maintaining portability to other Java EE containers. Also check out From Spring to Java EE 6 and Java EE 6 vs. Spring Framework: A technology decision making process. * Developer Productivity Report – Part 2: Developer Stress: A report on what stresses developers (things like deadlines and perfomance issues reached the top) and an interview with Martijn Verburg who shares his opinion on the matter. I really liked his approach on keeping up with new technologies (identify trends and learn the principles behind them). * Three Pillars of Project Management: This article discusses the tree main pillars of project management, i.e. (domain) knowledge, experience in the field and people skills. A synergy of all three elements is key for a successful career as a project manager. * A Basic Guide To Passwords and Social Engineering: As the title suggests, this is a rudimentary guide to passwords and social engineering. Provides some examples and use cases and suggests a few tips to enhance security. Also check Simple Security Rules and Hints for writing secure code. That’s all for this week. Stay tuned for more, here at Java Code Geeks. Cheers, Ilias Tsagklis...
packt-logo

Win Free Copies of Android books

Fellow Java geeks, We are pleased to announce that we have teamed up with Packt Publishing and we are organizing a Giveaway for you! 3 lucky winners stand a chance to win 2 copies of Packt’s best-selling books on Android (ebook format) each. Keep reading to find out how you can be one of the lucky winners… The Prizes 1. Android 3.0 Animations: Beginner’s GuideThe first and only book dedicated to creating animations for Android apps. Covers all of the commonly used animation techniques for Android 3.0 and lower versions. A step-by-step guide for learning animation by building fun example applications and games.Read more about this book and download free Sample Chapter 2. Android NDK Beginner’s GuideCreate high performance applications with C/C++ and integrate with Java Exploit advanced Android features such as graphics, sound, input and sensing Port and reuse your own or third-party libraries from the prolific C/C++ ecosystemRead more about this book and download free Sample Chapter How to Enter? Just send an email here using as subject “Packt Android Giveaway“. An empty email will do, It’s that simple! (Note: By entering the contest you will be automatically included in the forthcoming Java Code Geeks Newsletter.) Deadline The contest will close on Monday 20 February 2012  PT. Winners will be contacted by email, so be sure to use your real email address! Important Notes… Please spread the news! The larger the number of participants the more people will be eligible for getting discounts on best-selling Packt’s books! Providing comments about what interests you the most about these books will be highly appreciated. Good Luck! The Java Code Geeks Team...
java-duke-logo

Building Games Using the MVC Pattern – Tutorial and Introduction

One useful architecture pattern in game development is the MVC (model-view-controller) pattern. It helps separate the input logic, the game logic and the UI (rendering). The usefulness is quickly noticeable in the early stages of any game development project because it allows to change things quickly without too much rework of code in all layers of the application.The following diagram is the simplest logical representation of the model view controller concept.Model-View-Controller patternExample Usage In an example game where the player controls a robot the following can happen:1 – User clicks/taps somewhere on the screen. 2 – The controller handles the click/tap and converts the event into an appropriate action. For example if the terrain is occupied by an enemy, an attack action is created, if it is empty terrain, then a move action is created and finally if the place where the user tapped is occupied by an obstacle, do nothing. 3 – The controller updates the robot‘s (model‘s) state accordingly. If the move action was created, then it changes the position, if the attack, then it fires. 4 – The renderer (view) gets notified about the state changes and renders the world’s current state.What this all means is, that the models (robots) don’t know anything about how to draw themselves, or how to change their state (position, hit points). They are dumb entities. In Java they are also called POJOs (plain old java objects). The controller is in charge of changing the models’ state and notify the renderer. The renderer has to have a reference to the models (robots and any other entities) and their state, in order to draw them. We know from the typical game architecture that the main loop acts as a super controller, which updates the states and then renders the objects onto the screen many times a second. We can put all the update and rendering into the main loop along with the robots but that would be messy. Let’s identify the different aspects (concerns) of our games. The modelsThe droid controlled by the player An arena where the droid can move Some obstacles Some enemies to shoot atThe controllersThe main loop and the input handler Controller to process player input Controller to perform actions on the player’s robot (move, attack)The viewsThe world renderer – to render the objects onto the screenCreating the Project For simplicity I have chosen the applet this time and will try to keep it very brief. The project has the following structure:MVC – Project structureThe file Droids.java is the applet and contains the main loop. package net.obviam.droids;import java.applet.Applet; import java.awt.Color; import java.awt.Event; import java.awt.Graphics; import java.awt.image.BufferedImage;public class Droids extends Applet implements Runnable {private static final long serialVersionUID = -2472397668493332423L;public void start() { new Thread(this).start(); }public void run() {setSize(480, 320); // For AppletViewer, remove later.// Set up the graphics stuff, double-buffering. BufferedImage screen = new BufferedImage(480, 320, BufferedImage.TYPE_INT_RGB); Graphics g = screen.getGraphics(); Graphics appletGraphics = getGraphics();long delta = 0l;// Game loop. while (true) { long lastTime = System.nanoTime();g.setColor(Color.black); g.fillRect(0, 0, 480, 320);// Draw the entire results on the screen. appletGraphics.drawImage(screen, 0, 0, null);// Lock the frame rate delta = System.nanoTime() - lastTime; if (delta < 20000000L) { try { Thread.sleep((20000000L - delta) / 1000000L); } catch (Exception e) { // It's an interrupted exception, and nobody cares } } if (!isActive()) { return; } } }public boolean handleEvent(Event e) { return false; } }Running the above code as an applet does nothing more than to sett up the main loop and to paint the screen black. There are 3 packages in the structure and the respective components will go there. net.obviam.droids.model will contain all the models net.obviam.droids.view will contain all the renderers net.obviam.droids.controller will contain all the controllers Creating the models The Droid Droid.java package net.obviam.droids.model;public class Droid {private float x; private float y; private float speed = 2f; private float rotation = 0f; private float damage = 2f;public float getX() { return x; } public void setX(float x) { this.x = x; } public float getY() { return y; } public void setY(float y) { this.y = y; } public float getSpeed() { return speed; } public void setSpeed(float speed) { this.speed = speed; } public float getRotation() { return rotation; } public void setRotation(float rotation) { this.rotation = rotation; } public float getDamage() { return damage; } public void setDamage(float damage) { this.damage = damage; } }It is a simple java object without any knowledge of the surrounding world. It has a position, rotation, speed and damage. These states are defined by the member variables and are accessible through the getter and setter methods. The game requires a few more models: obstacles and enemies on a map. For simplicity the obstacles will have just a position on the map and the enemies will be standing objects. The map will be a 2 dimensional array holding the enemies, obstacles and the droid. The map will be called Arena to differentiate from the standard Java map and will be populated with obstacles and enemies when it is constructed. Obstacle.java package net.obviam.droids.model;public class Obstacle {private float x; private float y;public Obstacle(float x, float y) { this.x = x; this.y = y; }public float getX() { return x; } public float getY() { return y; } }Enemy.java package net.obviam.droids.model;public class Enemy {private float x; private float y; private int hitpoints = 10;public Enemy(float x, float y) { this.x = x; this.y = y; }public float getX() { return x; } public float getY() { return y; } public int getHitpoints() { return hitpoints; } public void setHitpoints(int hitpoints) { this.hitpoints = hitpoints; } }Arena.java package net.obviam.droids.model;import java.util.ArrayList; import java.util.List; import java.util.Random;public class Arena {public static final int WIDTH = 480 / 32; public static final int HEIGHT = 320 / 32;private static Random random = new Random(System.currentTimeMillis());private Object[][] grid; private List<Obstacle> obstacles = new ArrayList<Obstacle>(); private List<Enemy> enemies = new ArrayList<Enemy>(); private Droid droid;public Arena(Droid droid) { this.droid = droid;grid = new Object[HEIGHT][WIDTH]; for (int i = 0; i < WIDTH; i++) { for (int j = 0; j < HEIGHT; j++) { grid[j][i] = null; } } // add 5 obstacles and 5 enemies at random positions for (int i = 0; i < 5; i++) { int x = random.nextInt(WIDTH); int y = random.nextInt(HEIGHT); while (grid[y][x] != null) { x = random.nextInt(WIDTH); y = random.nextInt(HEIGHT); } grid[y][x] = new Obstacle(x, y); obstacles.add((Obstacle) grid[y][x]); while (grid[y][x] != null) { x = random.nextInt(WIDTH); y = random.nextInt(HEIGHT); } grid[y][x] = new Enemy(x, y); enemies.add((Enemy) grid[y][x]); } }public List<Obstacle> getObstacles() { return obstacles; } public List<Enemy> getEnemies() { return enemies; } public Droid getDroid() { return droid; } }The Arena is a more complex object but reading through the code should make it easy to understand. It basically groups all the models together into a single world. Our game world is the arena which contains all the elements like our droid, enemies and obstacles. The WIDTH and HEIGHT are calculated based on the resolution I have chosen. A cell (tile) on the grid will be 32 pixels wide and tall so I just compute how many cells go onto the grid. In the constructor (line #19) the grid is set up and 5 obstacles and 5 enemies are randomly placed. This will make up the starting arena and our game world. In order to keep the main loop tidy, we’ll delegate the update and rendering to the GameEngine. This is a simple class that will process the user input, update the states of the models and will render the world. It is a tiny glue framework to make all these happen. The GameEngine.java stub package net.obviam.droids.controller;import java.awt.Event; import java.awt.Graphics;public class GameEngine {/** handle the Event passed from the main applet **/ public boolean handleEvent(Event e) { switch (e.id) { case Event.KEY_PRESS: case Event.KEY_ACTION: // key pressed break; case Event.KEY_RELEASE: // key released break; case Event.MOUSE_DOWN: // mouse button pressed break; case Event.MOUSE_UP: // mouse button released break; case Event.MOUSE_MOVE: // mouse is being moved break; case Event.MOUSE_DRAG: // mouse is being dragged (button pressed) break; } return false; }/** the update method with the deltaTime in seconds **/ public void update(float deltaTime) { // empty }/** this will render the whole world **/ public void render(Graphics g) { // empty } }To use the engine the Droids.java class needs to be modified. We need to create an instance of the GameEngine class and call the update() and render() methods at the appropriate times. Also we need to delegate the input processing to the engine. Add the following lines: Declare the private member and also instantiate it. private GameEngine engine = new GameEngine();The modified game loop looks like this: while (true) { long lastTime = System.nanoTime();g.setColor(Color.black); g.fillRect(0, 0, 480, 320);// Update the state (convert to seconds) engine.update((float)(delta / 1000000000.0)); // Render the world engine.render(g);// Draw the entire results on the screen. appletGraphics.drawImage(screen, 0, 0, null);// Lock the frame rate delta = System.nanoTime() - lastTime; if (delta < 20000000L) { try { Thread.sleep((20000000L - delta) / 1000000L); } catch (Exception e) { // It's an interrupted exception, and nobody cares } } }The highlighted lines (#7-#10) contain the delegation to the update() and render() methods. Note that there is a conversion to seconds from nano seconds. It’s very useful to work in seconds as we can work with real world values. Important: The update needs to happen after the delta (time elapsed since the last update) has been calculated. Also the render should be called after the update so it will display the current state of the objects. Note that the screen is cleared each time before the render (painted black). The last thing to be done is to delegate the input handling. Replace the current handleEvent method with the following snippet: public boolean handleEvent(Event e) { return engine.handleEvent(e); }Very simple straightforward delegation. Running the applet yields no particular exciting result. Just a black screen. It makes sense as everything is just a stub apart from the screen being cleared every cycle. Initialising the models (world) Our game needs a droid and some enemies. By design, the world is our Arena. By instantiating it, we have created a world (check the constructor in Arena). We will create the world in the GameEngine as the engine is responsible for telling the view what to render. We also need the Droid to be created here because the Arena requires it its constructor. Is good to have it separate as the droid will be controlled by the player. Add the following members to the GameEngine along with the constructor which initialises the world. private Arena arena; private Droid droid;public GameEngine() { droid = new Droid(); // position droid in the middle droid.setX(Arena.WIDTH / 2); droid.setY(Arena.HEIGHT / 2); arena = new Arena(droid); }Note: Arena‘s constructor needs to be modified, so the Droid gets added to the grid before the obstacles and enemies. ... // add the droid grid[(int)droid.getY()][(int) droid.getX()] = droid; ...Running the applet again, won’t change the output but we have our world created. We can add logging to see the result but that won’t be interesting. Let’s create our first view which will reveal our world. Creating the first View/Renderer We’ve put a lot of effort in creating the arena and world, we’re eager to actually see it. Because of this, we will create a quick and dirty renderer to reveal the world. By quick and dirty I mean no fancy images or anything but simple squares, circles and placeholders. Once we are happy that the game elements are in we can work on a more elaborate view to replace the squares and circles with fancy graphics. This is where the power of decoupling shines. Steps to render the world.Draw the grid to see where the cells are. Obstacles will be drawn as blue squares and they will occupy the cells Enemies will be red circles The droid will be a green circle with a brown squareFirst we create the renderer interface. We use this to establish a single way to interact with the renderer and it will make it easy to create more views without affecting the game engine. To read more on why is a good idea check this and this. Create an interface in the view package. Renderer.java package net.obviam.droids.view;import java.awt.Graphics;public interface Renderer { public void render(Graphics g); }That is all. It contains one single method: render(Graphics g). The Graphics g is the canvas that is passed from the applet. Ideally the interface will be agnostic of this and each implementation will use a different back-end but the purpose of this exercise is to describe the MVC not to create a full framework. Because we have chosen applet we need the Graphics object. The concrete implementation looks like this: SimpleArenaRenderer.java (in the view package) package net.obviam.droids.view;import java.awt.Color; import java.awt.Graphics;import net.obviam.droids.model.Arena; import net.obviam.droids.model.Droid; import net.obviam.droids.model.Enemy; import net.obviam.droids.model.Obstacle;public class SimpleArenaRenderer implements Renderer {private Arena arena;public SimpleArenaRenderer(Arena arena) { this.arena = arena; }@Override public void render(Graphics g) { // render the grid int cellSize = 32; // hard coded g.setColor(new Color(0, 0.5f, 0, 0.75f)); for (int i = 0; i <= Arena.WIDTH; i++) { g.drawLine(i * cellSize, 0, i * cellSize, Arena.HEIGHT * cellSize); if (i <= Arena.WIDTH) g.drawLine(0, i * cellSize, Arena.WIDTH * cellSize, i * cellSize); }// render the obstacles g.setColor(new Color(0, 0, 1f)); for (Obstacle obs : arena.getObstacles()) { int x = (int) (obs.getX() * cellSize) + 2; int y = (int) (obs.getY() * cellSize) + 2; g.fillRect(x, y, cellSize - 4, cellSize - 4); }// render the enemies g.setColor(new Color(1f, 0, 0)); for (Enemy enemy : arena.getEnemies()) { int x = (int) (enemy.getX() * cellSize); int y = (int) (enemy.getY() * cellSize); g.fillOval(x + 2, y + 2, cellSize - 4, cellSize - 4); }// render player droid g.setColor(new Color(0, 1f, 0)); Droid droid = arena.getDroid(); int x = (int) (droid.getX() * cellSize); int y = (int) (droid.getY() * cellSize); g.fillOval(x + 2, y + 2, cellSize - 4, cellSize - 4); // render square on droid g.setColor(new Color(0.7f, 0.5f, 0f)); g.fillRect(x + 10, y + 10, cellSize - 20, cellSize - 20); } }Lines #13 – #17 declare the Arena object and make sure that it is set when the renderer is constructed. I called it ArenaRenderer because we will render the arena (world). The only method in the renderer is the render() method. Let’s see what it does step by step. #22 – Declare a cell size in pixels. It is 32. It’s hard coded as in the Arena class. #23 – #28 – The grid is being drawn. It is a simple grid. First the colour is set to dark green and lines are drawn at equal distance. Drawing the obstacles – blue squares #31 – Set the brush color to blue. #32 – #36 – Iterate through all the obstacles in the arena and for each it draws a blue filled rectangle slightly smaller than the cell on the grid. #39 – #44 – Sets the color to red and by iterating through the enemies in the arena, it draws a circle at the respective position. #47 – #54 – Finally draws the droid as a green circle with a brown square on top.Note that the arena in the real world has a width of 15 (480 / 32). So the droid will always be at the same position (7, 5) and the renderer works out its position on the screen by using a unit measure conversion. In this case is 1 unit in world coordinate is 32 pixels on the screen. By modifying the GameEngine to use the newly created view (SimpleArenaRenderer) we get the result. public class GameEngine {private Arena arena; private Droid droid; private Renderer renderer;public GameEngine() { droid = new Droid(); // position droid in the middle droid.setX(Arena.WIDTH / 2); droid.setY(Arena.HEIGHT / 2); arena = new Arena(droid);// setup renderer (view) renderer = new SimpleArenaRenderer(arena); }/** ... code stripped ... **//** this will render the whole world **/ public void render(Graphics g) { renderer.render(g); } }Pay attention to the highlighted lines (5, 15, 22). These are the lines where the renderer (view) is added to the game. The result should look like the following image (the positions are random apart from the player’s droid):The result of the first viewThis is a great view to test out the arena and see the models. It’s extremely easy to create a new view that instead of shapes (squares and circles) displays actual sprites. Controller for Handling Input and Update Models So far the game does nothing but displays the current state of the world (arena). For simplicity we will update just one state of the droid, its position. The steps to move the droid based on user input are:On mouse up check if the clicked cell on the grid is empty. This means that it does contain any objects which can be Enemy or Obstacle instances. If the cell is empty, the controller will create an action that will move the droid at a constant speed until it reaches the destination.package net.obviam.droids.controller;import net.obviam.droids.model.Arena; import net.obviam.droids.model.Droid;public class ArenaController {private static final int unit = 32; private Arena arena;/** the target cell **/ private float targetX, targetY; /** true if the droid moves **/ private boolean moving = false;public ArenaController(Arena arena) { this.arena = arena; }public void update(float delta) { Droid droid = arena.getDroid(); if (moving) { // move on X int bearing = 1; if (droid.getX() > targetX) { bearing = -1; } if (droid.getX() != targetX) { droid.setX(droid.getX() + bearing * droid.getSpeed() * delta); // check if arrived if ((droid.getX() < targetX && bearing == -1) || (droid.getX() > targetX && bearing == 1)) droid.setX(targetX); } // move on Y bearing = 1; if (droid.getY() > targetY) { bearing = -1; } if (droid.getY() != targetY) { droid.setY(droid.getY() + bearing * droid.getSpeed() * delta); // check if arrived if ((droid.getY() < targetY && bearing == -1) || (droid.getY() > targetY && bearing == 1)) droid.setY(targetY); } // check if arrived if (droid.getX() == targetX && droid.getY() == targetY) moving = false; } }/** triggered with the coordinates every click **/ public boolean onClick(int x, int y) { targetX = x / unit; targetY = y / unit; if (arena.getGrid()[(int) targetY][(int) targetX] == null) { // start moving the droid towards the target moving = true; return true; } return false; } }The following breakdown explains the logic and important bits. #08 – The unit represents how many pixels are in a cell which represents 1 unit in world coordinates. It’s hard-coded and not optimal but for the demo is good enough. #09 – The Arena the controller will control. It is set when the controller is constructed (line #16). #12 – The target coordinates of the click in world units. #14 – It is true when the droid is moving. This is the “move” actions’ state. Ideally this should be a stand alone class, but to demonstrate the controller and keep it concise, we’ll hack an action together inside the controller. #20 – The update method that updates the position of the droid according to the time passed at a constant speed. It is extremely simple, it checkes both X and Y positions and if they are not the same as the target position, it updates the droid’s respective position (X or Y) considering its speed. If the droid is in the target position, then the move state variable is updated completing the move action. This is not a very well written action, there is no collision checking on obstacles or enemies found along the way, no path finding. It just updates state. #52 – The onClick(int x, int y) method will be called when the “mouse up” event occurs. It checks if the clicked cell is empty and if so, then it starts the “move” action by setting the state variable to true #53-#54 – Converts screen coordinates to world coordinates. This is the controller. To use it, the GameEngine needs to be updated. The updated GameEngine.java package net.obviam.droids.controller;import java.awt.Event; import java.awt.Graphics;import net.obviam.droids.model.Arena; import net.obviam.droids.model.Droid; import net.obviam.droids.view.Renderer; import net.obviam.droids.view.SimpleArenaRenderer;public class GameEngine {private Arena arena; private Droid droid; private Renderer renderer; private ArenaController controller;public GameEngine() { droid = new Droid(); // position droid in the middle droid.setX(Arena.WIDTH / 2); droid.setY(Arena.HEIGHT / 2); arena = new Arena(droid);// setup renderer (view) renderer = new SimpleArenaRenderer(arena); // setup controller controller = new ArenaController(arena); }/** handle the Event passed from the main applet **/ public boolean handleEvent(Event e) { switch (e.id) { case Event.KEY_PRESS: case Event.KEY_ACTION: // key pressed break; case Event.KEY_RELEASE: // key released break; case Event.MOUSE_DOWN: // mouse button pressed break; case Event.MOUSE_UP: // mouse button released controller.onClick(e.x, e.y); break; case Event.MOUSE_MOVE: // mouse is being moved break; case Event.MOUSE_DRAG: // mouse is being dragged (button pressed) break; } return false; }/** the update method with the deltaTime in seconds **/ public void update(float deltaTime) { controller.update(deltaTime); }/** this will render the whole world **/ public void render(Graphics g) { renderer.render(g); } }The changes are highlighted. #16 – Declare the controller. #28 – Instantiate the controller. #46 – Delegating the mouse up event. #60 – Call the update method on the controller. Run the applet and you can click on the map and if the cell is empty, the droid will move there. ExcercisesCreate a view that will display images/sprites for entities instead of the drawn shapes. Hint: Use BufferedImage to achieve that.Extract the move action into a new class. Add new actions (attack) when an enemy is clicked Hint:Create a bullet entity that gets fired to the target. You can use the move action with a higher speed. When the hitpoint gets down to 0, then the enemy is destroyed. Use a different image to represent different states.Source Code https://github.com/obviam/mvc-droids or download as zip file. You can also use git $ git clone git://github.com/obviam/mvc-droids.git Reference: Building Games Using the MVC Pattern – Tutorial and Introduction from our JCG partner Impaler at the Against the Grain blog....
play-framework-logo

Play! Framework + Google Guice

In the project I’m currently working, we started to use Google Guice. For those who don’t know, Google Guice is a dependency injection framework. The basic idea behind dependency injection is to provide a class it’s dependencies, instead of making the dependent class responsible of instantiating the objects on which it depends. Play has a module for integrating Guice: http://www.playframework.org/modules/guice-1.2/home In addition to the module documentation, this post from @_felipera can help you get started. http://geeks.aretotally.in/dependency-injection-with-play-framework-and-google-guice How to use the Guice module Add the dependency require: - play - play -> guice 1.2Download the dependencies play depsCreate a new class that will be injected in the controller services.MyService package services; public interface MyService { public void sayHello(); }services.MyServiceImpl package services; public class MyServiceImpl implements MyService { public MyServiceImpl(){ play.Logger.info("constructor!"); }@Override public void sayHello() { play.Logger.info("hello"); } }Configure the injector package config; public class GuiceConfig extends GuiceSupport { @Override protected Injector configure() { return Guice.createInjector(new AbstractModule() { @Override protected void configure() { bind(MyService.class).to(MyServiceImpl.class).in(Singleton.class); } }); } }This will setup the class as a singleton. Each time a class has the dependency of MyService, the injector will inject the same instance of MyServiceImpl. Inject the dependency using the @Inject annotation package controllers; public class Application extends Controller {@Inject static MyService myService;public static void index() { myService.sayHello(); render(); } }Testing My next step was to create a test and here came the first problem play testhttp://localhost:9000/@tests Compilation error! The problem is that the module has a folder called ‘test’. This folder should have some unit or functional test, but instead it has three sample applications. The convention in play modules is to have these kind of applications in the ‘samples-and-test’ folder. I made a fork of the project to rename this folder: https://github.com/axelhzf/play-guice-module I also did a pull-request but I didn’t get any response so far : https://github.com/pk11/play-guice-module/pull/5 Renaming the folder ‘test’ was enough to run this test: @InjectSupport public class InjectTest extends UnitTest { @Inject static MyService myService;@Test public void injectOk(){ assertNotNull(myService); } }Adding more dependencies By default, Play automatically detects the @Inject annotation on classes than inherit from Controller, Job and Mail. If you want to inject dependencies on other classes you must use the @InjectSupport. Typically our services are not as simple as MyService. It is common to have dependencies between services. Guice solves this problem analyzing the dependencies and instantiating the objects in the proper order. services.MyDependentService package services;public interface MyDependentService { public void sayHelloWorld(); }service.MyDependentServiceImpl package services;@InjectSupport public class MyDependentServiceImpl implements MyDependentService { @Inject static MyService myService;public MyDependentServiceImpl(){ play.Logger.info("Init MyDependentServiceImpl"); }public void sayHelloWorld(){ myService.sayHello(); play.Logger.info("world"); } }InjectTest @InjectSupport public class InjectTest extends UnitTest {@Inject static MyDependentService myDependentService;@Test public void injectOk(){ assertNotNull(myDependentService); myDependentService.sayHelloWorld(); }}Binding in GuiceConfig bind(MyDependentService.class).to(MyDependentServiceImpl.class).in(Singleton.class);And this is the console output 20:34:39,090 INFO ~ Init MyServiceImpl 20:34:39,095 INFO ~ Init MyDependentServiceImpl 20:34:39,095 INFO ~ Application 'lazySingleton' is now started ! 20:34:39,136 INFO ~ hello 20:34:39,136 INFO ~ worldConstructor injection One of the things I don’t like about the module is that the fields you are allowed to inject must be static. I would prefer to declare the dependencies as constructor parameters. This way, it would be more obvious that to create an instance of MyDependentServiceImpl you need a MyService instance. Moreover, when working on units test, it’s easier to pass a mock objects as a parameter than to configure an injector. In the module documentation I didn’t found any reference to how to do it. Nevertheless, I found an article explaining how to do this using a Provider: http://ericlefevre.net/wordpress/2011/05/08/play-framework-and-guice-use-providers-in-guice-modules/ Later, I found a question on StackOverflow that gave me another clue: http://stackoverflow.com/questions/8435686/does-injector-getinstance-always-call-a-constructor In the Edit he says that he forgot to put the @Inject annotation in the constructor. I tried to do the same thing and finally it worked: public class MyDependentServiceImpl implements MyDependentService {private final MyService myService; @Injectpublic MyDependentServiceImpl(MyService myService){ this.myService = myService; play.Logger.info("Inicializando MyDependentServiceImpl"); }...Lazy Singletons It still remains one last detail to have a perfect google guice configuration. The services are initialized when the application starts. 21:38:11,801 INFO ~ Inicializando MyServiceImpl 21:38:11,805 INFO ~ Inicializando MyDependentServiceImpl 21:38:11,805 INFO ~ Application 'lazySingleton' is now started !When the application is in production mode this is the correct behavior. But in development mode I prefer that Singletons are intialized on demand. There may be services that take their time to start and I want the application startup to be as fast as possible. With Google Guice you can achieve this using Scopes: http://code.google.com/p/google-guice/wiki/Scopes All you have to do is to set the Stage parameter: Stage stage = Play.mode.isDev() ? Stage.DEVELOPMENT : Stage.PRODUCTION; return Guice.createInjector(stage, new AbstractModule() {…..Rerunning the test 22:00:03,353 WARN ~ You're running Play! in DEV mode 22:00:04,615 INFO ~ Connected to jdbc:h2:mem:play;MODE=MYSQL;LOCK_MODE=0 22:00:04,811 INFO ~ Guice injector created: config.GuiceConfig 22:00:04,819 INFO ~ Init MyServiceImpl 22:00:04,824 INFO ~ Init MyDependentServiceImpl 22:00:04,824 INFO ~ Application 'lazySingleton' is now started !Ooops! The Singleton are initialized before application starts. Perhaps that’s not the correct use for the stage variable. Let’s try a test: public class StageTest {@Test public void testDevelopment(){ Injector injector = createInjector(Stage.DEVELOPMENT); System.out.println("development - before getInstance"); MyService instance = injector.getInstance(MyService.class); System.out.println("development - after getInstance"); }@Test public void testProduction(){ Injector injector = createInjector(Stage.PRODUCTION); System.out.println("production - before getInstance"); MyService instance = injector.getInstance(MyService.class); System.out.println("production - after getInstance"); }public Injector createInjector(Stage stage){ Injector injector = Guice.createInjector(stage, new AbstractModule(){ @Override protected void configure() { bind(MyService.class).to(MyServiceImpl.class); } }); return injector; } }And the result is: INFO: development - before getInstance INFO: Inicializando MyServiceImpl INFO: development - after getInstanceINFO: Inicializando MyServiceImpl INFO: production - before getInstance INFO: production - after getInstanceJust like it says on the documentation, in development mode Singletons are initialized lazily. If this works, when I tried using the play module why didn’t it work? Reviewing the code: https://github.com/pk11/play-guice-module/blob/master/src/play/modules/guice/GuicePlugin.java @OnApplicationStart the module finds all classes annotated with @InjectSupport and inject it’s dependencies. To inject the dependencies the module calls the getBean() method. And here we find the problem: when calling getBean() the class is instantiated. I found a solution to this problem: https://groups.google.com/d/msg/google-guice/405HVgnCzsQ/fBUuueP6NfsJ This is the code:@LazySingleton MoreScopes LazyBinderThese classes create a proxy for each class annotated as @LazySingleton. When the object is injected the injector actually injects the proxy. The first time a method is invoke, the proxy will take care of initializing the class. Using these classes, the injector configuration looks like: public class GuiceConfig extends GuiceSupport { @Override protected Injector configure() { Stage stage = Play.mode.isDev() ? Stage.DEVELOPMENT : Stage.PRODUCTION; return Guice.createInjector(stage, new AbstractModule() { @Override protected void configure() { bindScope(LazySingleton.class, MoreScopes.LAZY_SINGLETON); bindLazySingletonOnDev(MyService.class, MyServiceImpl.class); bindLazySingletonOnDev(MyDependentService.class, MyDependentServiceImpl.class); }protected &lt;T&gt; void bindLazySingletonOnDev(Class&lt;T&gt; expected, Class&lt;? extends T&gt; implClass){ if(Play.mode.isDev()){ bind(implClass).in(MoreScopes.LAZY_SINGLETON); Provider&lt;T&gt; provider = LazyBinder.newLazyProvider(expected, implClass); bind(expected).toProvider(provider); }else{ bind(expected).to(implClass).in(Scopes.SINGLETON); } } }); } }I will add this classes to the fork in order to have a complete module that can be reused across projects. Conclusion In the last few years Dependency Injection has has evolved from being an obscure and little understood term, to become part of every programmer’s toolchest. In this article we saw how easy is to integrate Guice, a very handy library from Google, into a play framework application. Moreover, we also covered how to customize it’s behaviour for a better development experience. Article original published at http://axelhzf.tumblr.com. Reference: Playframework + Google Guice from our JCG partner Sebastian Scarano at the Having fun with Play framework! blog....
json-logo

JSON web service with Java and Axis2

I have recently met a client who asking me to rebuild their legacy product using Java web service. They want it modular and easy to use. The first thing crossed my mind is using restful approach. But there is only thing that bothers me, Java restful approach is using XML!, I would prefer a simpler way to communicate, easy to understand and parsed data, so Jsonit is. I start google-ing a little and came across this excellent blog Json web service with Java and Axis2? href=”http://www.marcusschiesser.de/2009/01/building-a-json-web-service-with-java-and-axis2/”>Building a Json web service with Java and Axis2. I read the blog but it seem there some minor problem which they(he and the blog commenter) already solved. So I am not here to revise on anything just to sum it up so it will be easier for you to build a Json web service with Java and Axis2. I am using Java 1.7 and Apache Tomcat 7.0. The steps are not so different with my reference blog, but I have minor modification on the steps. 1. Download the Axis2 war that includes jetisson patch and DynamicRespondHandler Axis2 Module here Note that the Axis2 above is pre-configure,but still I would like to show what the changes are: – Added module reference to the axis2.xml file <br />&lt;module ref="DynamicResponseHandler"/&gt;<br /> - Added Json Message formatters <br />&lt;messageFormatter contentType="application/json" class="org.apache.axis2.json.JSONMessageFormatter"/&gt;<br />&lt;messageFormatter contentType="application/json/badgerfish" class="org.apache.axis2.json.JSONBadgerfishMessageFormatter"/&gt;<br /> - Added Json Message builders <br />&lt;messageBuilder contentType="application/json" class="org.apache.axis2.json.JSONOMBuilder"/&gt;<br />&lt;messageBuilder contentType="application/json/badgerfish" class="org.apache.axis2.json.JSONBadgerfishOMBuilder"/&gt;<br /> 2. Download you favorite Java application server, in this case I am using Apache Tomcat 7.0. 3. Deploy Axis2 war to your application server, in my case I just copy the Axis2 war to Tomcat`s webapps folder. 4. Start your application server and open this URL using your web browser (your URL might slightly different then mine, depend on your HTTP port setting on your application server):http://localhost:8080/axis2/services/Version/getVersion?response=application/json your browser should shows this:So that is the start, I hopefully can post more about this in the future. Cheers. Reference: JSON web service with Java and Axis2 from our JCG partner Djaka Pribadi Maulana at the A.K.A Self notes blog....
scala-logo

Implicit Conversions in Scala

Following on from the previous post on operator overloading I’m going to be looking at Implicit Conversions, and how we can combine them to with operator overloading to do some really neat things, including one way of creating a multi-parameter conversion. So what’s an “Implicit Conversion” when it’s at home? So lets start with some basic Scala syntax, if you’ve spent any time with Scala you’ve probably noticed it allows you to do things like: (1 to 4).foreach(println) // print out 1 2 3 4Ever wondered how it does this? Lets make things more explicit, you could rewrite the above code as: val a : Int = 1 val b : Int = 4 val myRange : Range = a to b myRange.foreach(println)Scala is creating a Range object directly from two Ints, and a method called to. So what’s going on here? Is this just a sprinkling of syntactic sugar to make writing loops easier? Is to just a keyword in like def or val? The answers to all this is no, there’s nothing special going on here. to is simply a method defined in the RichInt class, which takes a parameter and returns a Range object (specifically a subclass of Range called Inclusive). You could rewrite it as the following if you really wanted to: val myRange : Range = a.to(b)Hang on though, RichInt may have a “to” method but Int certainly doesn’t, in your example you’re even explicitly casting your numbers to Ints Which brings me nicely on to the subject of this post, Implicit Conversions. This is how Scala does this. Implicit Conversions are a set of methods that Scala tries to apply when it encounters an object of the wrong type being used. In the case of the to example there’s a method defined and included by default that will convert Ints into RichInts. So when Scala sees 1 to 4 it first runs the implicit conversion on the 1 converting it from an Int primitive into a RichInt. It can then call the to method on the new RichInt object, passing in the second Int (4) as the parameter. Hmm, think I understand, how’s about another example? Certainly. Lets try to improve our Complex number class we created in the previous post. Using operator overloading we were able to support adding two complex numbers together using the + operator. eg. class Complex(val real : Double, val imag : Double) { def +(that: Complex) = new Complex(this.real + that.real, this.imag + that.imag) def -(that: Complex) = new Complex(this.real - that.real, this.imag - that.imag) override def toString = real + " + " + imag + "i" } object Complex { def main(args : Array[String]) : Unit = { var a = new Complex(4.0,5.0) var b = new Complex(2.0,3.0) println(a) // 4.0 + 5.0i println(a + b) // 6.0 + 8.0i println(a - b) // 2.0 + 2.0i } }But what if we want to support adding a normal number to a complex number, how would we do that? We could certainly overload our “+” method to take a Double argument, ie something like… def +(n: Double) = new Complex(this.real + n, this.imag)Which would allow us to do… val sum = myComplexNumber + 8.5…but it’ll break if we try… val sum = 8.5 + myComplexNumberTo get around this we could use an Implicit Conversion. Here’s how we create one. object ComplexImplicits { implicit def Double2Complex(value : Double) = new Complex(value,0.0) }Simple! Although you do need to be careful to import the ComplexImplicits methods before they can be used. You need to make sure you add the following to the top of your file (even if your Implicits object is in the same file)… import ComplexImplicits._And that’s the problem solved, you can now write val sum = 8.5 + myComplexNumber and it’ll do what you expect! Nice. Is there anything else I can do with them? One other thing I’ve found them good for is creating easy ways of instantiating objects. Wouldn’t it be nice if there were a simpler way of creating one of our complex numbers other than with new Complex(3.0,5.0). Sure you could get rid of the new by making it a case class, or implementing an apply method. But we can do better, how’s about just (3.0,5.0) Awesome, but I’d need some sort of multi parameter implicit conversion, and I don’t really see how that’s possible!? The thing is, ordinarily (3.0,5.0) would create a Tuple. So we can just use that tuple as the parameter for our implicit conversion and convert it into a Complex. how we might go about doing this… implicit def Tuple2Complex(value : Tuple2[Double,Double]) = new Complex(value._1,value._2);And there we have it, a simple way to instantiate our Complex objects, for reference here’s what the entire Complex code looks like now. import ComplexImplicits._ object ComplexImplicits { implicit def Double2Complex(value : Double) = new Complex(value,0.0) implicit def Tuple2Complex(value : Tuple2[Double,Double]) = new Complex(value._1,value._2); } class Complex(val real : Double, val imag : Double) { def +(that: Complex) : Complex = (this.real + that.real, this.imag + that.imag) def -(that: Complex) : Complex = (this.real - that.real, this.imag + that.imag) def unary_~ = Math.sqrt(real * real + imag * imag) override def toString = real + " + " + imag + "i" } object Complex { val i = new Complex(0,1); def main(args : Array[String]) : Unit = { var a : Complex = (4.0,5.0) var b : Complex = (2.0,3.0) println(a) // 4.0 + 5.0i println(a + b) // 6.0 + 8.0i println(a - b) // 2.0 + 8.0i println(~b) // 3.60555 var c = 4 + b println(c) // 6.0 + 3.0i var d = (1.0,1.0) + c println(d) // 7.0 + 4.0i } }Reference: Implicit Conversions in Scala from our JCG partner Tom Jefferys at the Tom’s Programming Blog ....
java-logo

High performance libraries in Java

There is an increasing number of libraries which are described as high performance and have benchmarks to back that claim up. Here is a selection that I am aware of. Disruptor library – http://code.google.com/p/disruptor/ LMAX aims to be the fastest trading platform in the world. Clearly, in order to achieve this we needed to do something special to achieve very low-latency and high-throughput with our Java platform. Performance testing showed that using queues to pass data between stages of the system was introducing latency, so we focused on optimising this area. The Disruptor is the result of our research and testing. We found that cache misses at the CPU-level, and locks requiring kernel arbitration are both extremely costly, so we created a framework which has “mechanical sympathy” for the hardware it’s running on, and that’s lock-free.The 6 million TPS benchmark was measured on a 3Ghz dual-socket quad-core Nehalem based Dell server with 32GB RAM.http://martinfowler.com/articles/lmax.html Java Chronicle – https://github.com/peter-lawrey/Java-Chronicle This library is an ultra low latency, high throughput, persisted, messaging and event driven in memory database. The typical latency is as low as 16 nano-seconds and supports throughputs of 5-20 million messages/record updates per second. It uses almost no heap, trivial GC impact, can be much larger than your physical memory size (only limited by the size of your disk). and can be shared between processes with better than 1/10th latency of using Sockets over loopback. It can change the way you design your system because it allows you to have independent processes which can be running or not at the same time (as no messages are lost) This is useful for restarting services and testing your services from canned data. e.g. like sub-microsecond durable messaging. You can attach any number of readers, including tools to see the exact state of the data externally. e.g. You can use; od -t cx1 {file} to see the current state. Colt Matrix library – http://acs.lbl.gov/software/colt/ Scientific and technical computing, as, for example, carried out at CERN, is characterized by demanding problem sizes and a need for high performance at reasonably small memory footprint. There is a perception by many that the Java language is unsuited for such work. However, recent trends in its evolution suggest that it may soon be a major player in performance sensitive scientific and technical computing. For example, IBM Watson’s Ninja project showed that Java can indeed perform BLAS matrix computations up to 90% as fast as optimized Fortran. The Java Grande Forum Numerics Working Group provides a focal point for information on numerical computing in Java. With the performance gap steadily closing, Java has recently found increased adoption in the field. The reasons include ease of use, cross-platform nature, built-in support for multi-threading, network friendly APIs and a healthy pool of available developers. Still, these efforts are to a significant degree hindered by the lack of foundation toolkits broadly available and conveniently accessible in C and Fortran. The latest stable Colt release breaks the 1.9 Gflop/s barrier on JDK ibm-1.4.1, RedHat 9.0, 2x IntelXeon@2.8 GHz. Javolution – http://javolution.org/ Javolution real-time goals are simple: To make your application faster and more time predictable! That being accomplished through:High performance and time-deterministic (real-time) util / lang / text / io / xml base classes. Context programming in order to achieve true separation of concerns (logging, performance, etc). A testing framework addressing not only unit tests but also performance and regression tests as well. Straightforward and low-level parallel computing capabilities with ConcurrentContext. Struct and Union base classes for direct interfacing with native applications (e.g. C/C++). World’s fastest and first hard real-time XML marshalling/unmarshalling facility. Simple yet flexible configuration management of your application.Trove collections for primitives – http://trove.starlight-systems.com/ The Trove library provides high speed regular and primitive collections for Java. The GNU Trove library has two objectives:Provide “free” (as in “free speech” and “free beer”), fast, lightweight implementations of the java.util Collections API. These implementations are designed to be pluggable replacements for their JDK equivalents. Provide primitive collections with similar APIs to the above. This gap in the JDK is often addressed by using the “wrapper” classes (java.lang.Integer, java.lang.Float, etc.) with Object-based collections. For most applications, however, collections which store primitives directly will require less space and yield significant performance gains.MG4J: Managing Gigabytes for Java™ – http://mg4j.dsi.unimi.it/ MG4J (Managing Gigabytes for Java) is a free full-text search engine for large document collections written in Java. MG4J is a highly customisable, high-performance, full-fledged search engine providing state-of-the-art features (such as BM25/BM25F scoring) and new research algorithms. Other links : Overview of 8 performance libraries http://www.dzone.com/links/r/8_best_open_source_high_performance_java_collecti.html Sometimes collection classes in JDK may not sufficient. We may require some high performance hashtable, Bigarrays etc. Check out the list of open source high performance collection libraries. Serialization benchmark http://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking This is a comparison of some serialization libraries. Its still hard to beat hand coded serialization. http://vanillajava.blogspot.com/2011/10/serialization-using-bytebuffer-and.html Reference: High performance libraries in Java from our JCG partner Peter Lawrey at the Vanilla Java blog....
jaspersoft-ireport-logo

Big Data analytics with Hive and iReport

Each J.J. Abrams’ TV series Person of Interest episode starts with the following narration from Mr. Finch one of the leading characters: “You are being watched. The government has a secret system–a machine that spies on you every hour of every day. I know because…I built it.” Of course us technical people know better. It would take a huge team of electrical and software engineers many years to build such a high performing machine and the budget would be unimaginable… or wouldn’t be? Wait a second we have Hadoop! Now everyone of us can be Mr. Finch for a modest budget thanks to Hadoop. In JCG article “Hadoop Modes Explained – Standalone, Pseudo Distributed, Distributed” JCG partner Rahul Patodi explained how to setup Hadoop. The Hadoop project has produced a lot of tools for analyzing semi-structured data but Hive is perhaps the most intuitive among them as it allows anyone with an SQL background to submit MapReduce jobs described as SQL queries. Hive can be executed from a command line interface, as well as run in a server mode with a Thrift client acting as a JDBC/ODBC interface giving access to data analysis and reporting applications. In this article we will set up a Hive Server, create a table, load it with data from a text file and then create a Jasper Resport using iReport. The Jasper Report executes an SQL query on the Hive Server that is then translated to a MapReduce job executed by Hadoop. Note: I used Hadoop version 0.20.205, Hive version 0.7.1 and iReport version 4.5 running OpenSuSE 12.1 Linux with MySQL 5.5 installed. Assuming you have already installed Hadoop download and install Hive following the Hive Getting Started wiki instructions. By default Hive is installed in CLI mode running on a Standalone Hadoop mode. Making a multiuser Hive metastore The default Hive install uses a derby embedded database as its metastore. The metastore is where Hive maintains descriptions of the data we want to access via SQL. In order for the metastore to be accessible from many users simultaneously it is necessary to be moved into a standalone database. Here is how to install a MySQL metastore.Copy the MySQL JDBC driver jar file to ~/hive-0.7.1-bin/lib directoryChange the following properties in file hive-default.xml found in ~/hive-0.7.1-bin/conf directory: <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://hyperion/metastore?createDatabaseIfNotExist=true</value> <description>JDBC connect string for a JDBC metastore</description> </property><property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> <description>Driver class name for a JDBC metastore</description> </property><property> <name>javax.jdo.option.ConnectionUserName</name> <value>foo</value> <description>Username to connect to the database</description> </property><property> <name>javax.jdo.option.ConnectionPassword</name> <value>bar</value> <description>Password to connect to the database</description> </property>Use MySQL workbench or the MySQL command line utility to create a schema using the latin1 character set. If Hive does not find a schema it will create it on its own using the default character set for MySQL. In my case that was UTF-8 and that generated jdbc errors. If you want to use the command line utility just type: mysql> CREATE DATABASE IF NOT EXISTS `metastore` DEFAULT CHARACTER SET latin1 COLLATE latin1_bin;Type Hive in your command prompt to enter Hive CLI and type: hive> SHOW TABLES; OK testlines Time taken: 3.654 seconds hive>That will populate your newly created metastore schema. If you see any errors check your hive-default.xml configuration and make sure your database schema is named ‘metastore’ with latin1 as the default Character Set.Now let’s populate Hadoop Hive with some data Let’s just create two text files named file01 and file02 each containing: file01: Hello World Bye World Hello Everybody Bye Everybody file02: Hello Hadoop Goodbye Hadoop Hello Everybody Goodbye Everybody Copy these files from your local filesystem to HDFS: $ hadoop fs -mkdir HiveExample $ hadoop fs -copyFromLocal ~/file* /user/ssake/HiveExampleGo to Hive CLI and create a table named testlines that would contain each line’s words in an array of strings: hive> create table testlines (line array<string>) row format delimited collection items terminated by ' ';Load the text files into Hive: hive> load data inpath "/user/ssake/HiveExample/file01" INTO table testlines; hive> load data inpath "/user/ssake/HiveExample/file02" INTO table testlines;Check that testlines now contains each line’s words: hive> select * from testlines; OK ["Hello","World","Bye","World"] ["Hello","Everybody","Bye","Everybody"] ["Hello","Hadoop","Goodbye","Hadoop"] ["Hello","Everybody","Goodbye","Everybody"] Time taken: 0.21 secondsNow that we have a Hive with data we can run it as a server in port 10000 which is typical for running a hive server: $ HIVE_PORT=10000 $ hive --service hiveserverWith this setup it is possible to have several Thrift clients accessing our Hive server. However according to the Apache Hive blog the multithreaded Hive features are not thoroughly tested and thus it is safer to use a separate port and hive instance per Thrift client. Create a “word count” report iReport 4.5 supports hive datasources so let’s use it to create a report that runs with our hive server as its datasource: 1. Create a datasource connecting to the hive server2. Use the report wizard to generate your report3. Type the following in the HiveQL Query input box: select word,count(word) from testlines lateral view explode(line) words as word group by wordLets briefly explain what the above query does: Our source table, the “testlines” table, has a single column named “line” which contains data in the form of array of strings. Each array of strings represents the words in a sentence as found in the imported files “file01” and “file02“. In order to properly count the occurrences of each distinct word in all of the input files we have to “explode” the arrays of strings from our source table into a new one that should contain every individual word. To do so we use the “lateral view” in conjunction with the “explode()” HiveQL commands as shown above. Issuing the HiveQL query above we create an new iconic table named “words” that has a single column named “word” containing all the words found in every string array from our “testlines” table.4. Click the … button to select all fileds and click next5. When you are at the designer view click the Preview tab to execute your HiveQL reportAnd here is our report:Now you are all set to build applications that access your Hadoop data using the familiar JDBC interface! Reference: Big Data analytics with Hive and iReport from our W4G partner Spyros Sakellariou....
codehaus-jetty-logo

Embedded Jetty, Vaadin and Weld

When I develop web applications I like to be able to quickly start them from Eclipse without having to rely on all kinds of heavy-weight tomcat or glassfish plugins. So what I often do is just create a simple Java based Jetty launcher I can run directly from Eclipse. This launcher starts up within a couple of seconds so this makes developing a lot more pleasant. However, sometimes, getting everything setup correctly is a bit hard. So in this article I’ll give you a quick overview of how you can setup Jetty together with Weld for CDI and Vaadin as the web framework. To get everything setup correctly we’ll need to take the following steps:Setup the maven pom for the required dependencies Create a java based Jetty Launcher Setup web.xml Add Weld placeholdersSetup the maven pom for the required dependencies I use the following pom.xml file. You might not need everything if you for instance don’t use custom components. But it should serve as a good reference of what should be in there. <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>group.id</groupId> <artifactId>artifact.id</artifactId> <packaging>war</packaging> <version>1.0</version> <name>Vaadin Web Application</name> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <vaadin.version>6.7.1</vaadin.version> <gwt.version>2.3.0</gwt.version> <gwt.plugin.version>2.2.0</gwt.plugin.version> </properties> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.5</source> <target>1.5</target> </configuration> </plugin> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>gwt-maven-plugin</artifactId> <version>${gwt.plugin.version}</version> <configuration> <webappDirectory>${project.build.directory}/${project.build.finalName}/VAADIN/widgetsets</webappDirectory> <extraJvmArgs>-Xmx512M -Xss1024k</extraJvmArgs> <runTarget>cvgenerator-web</runTarget> <hostedWebapp>${project.build.directory}/${project.build.finalName}</hostedWebapp> <noServer>true</noServer> <port>8080</port> <compileReport>false</compileReport> </configuration> <executions> <execution> <goals> <goal>resources</goal> <goal>compile</goal> </goals> </execution> </executions> <dependencies> <dependency> <groupId>com.google.gwt</groupId> <artifactId>gwt-dev</artifactId> <version>${gwt.version}</version> </dependency> <dependency> <groupId>com.google.gwt</groupId> <artifactId>gwt-user</artifactId> <version>${gwt.version}</version> </dependency> </dependencies> </plugin> <plugin> <groupId>com.vaadin</groupId> <artifactId>vaadin-maven-plugin</artifactId> <version>1.0.2</version> <executions> <execution> <configuration> </configuration> <goals> <goal>update-widgetset</goal> </goals> </execution> </executions> </plugin> </plugins> </build> <!-- extra repositories for Vaadin extensions --> <repositories> <repository> <id>vaadin-snapshots</id> <url>http://oss.sonatype.org/content/repositories/vaadin-snapshots/</url> <releases> <enabled>false</enabled> </releases> <snapshots> <enabled>true</enabled> </snapshots> </repository> <repository> <id>vaadin-addons</id> <url>http://maven.vaadin.com/vaadin-addons</url> </repository> </repositories> <!-- repositories for the plugins --> <pluginRepositories> <pluginRepository> <id>codehaus-snapshots</id> <url>http://nexus.codehaus.org/snapshots</url> <snapshots> <enabled>true</enabled> </snapshots> <releases> <enabled>false</enabled> </releases> </pluginRepository> <pluginRepository> <id>vaadin-snapshots</id> <url>http://oss.sonatype.org/content/repositories/vaadin-snapshots/</url> <snapshots> <enabled>true</enabled> </snapshots> <releases> <enabled>false</enabled> </releases> </pluginRepository> </pluginRepositories> <!-- minimal set of dependencies --> <dependencies> <dependency> <groupId>com.vaadin</groupId> <artifactId>vaadin</artifactId> <version>${vaadin.version}</version> </dependency> <dependency> <groupId>org.vaadin.addons</groupId> <artifactId>stepper</artifactId> <version>1.1.0</version> </dependency> <!-- the jetty version we'll use --> <dependency> <groupId>org.eclipse.jetty.aggregate</groupId> <artifactId>jetty-all-server</artifactId> <version>8.0.4.v20111024</version> <type>jar</type> <scope>compile</scope> <exclusions> <exclusion> <artifactId>mail</artifactId> <groupId>javax.mail</groupId> </exclusion> </exclusions> </dependency> <!-- vaadin custom field addon --> <dependency> <groupId>org.vaadin.addons</groupId> <artifactId>customfield</artifactId> <version>0.9.3</version> </dependency> <!-- with cdi utils plugin you can use Weld --> <dependency> <groupId>org.vaadin.addons</groupId> <artifactId>cdi-utils</artifactId> <version>0.8.6</version> </dependency> <!-- we'll use this version of Weld --> <dependency> <groupId>org.jboss.weld.servlet</groupId> <artifactId>weld-servlet</artifactId> <version>1.1.5.Final</version> <type>jar</type> <scope>compile</scope> </dependency> <!-- normally following are provided, but not if you run within jetty --> <dependency> <groupId>javax.servlet</groupId> <artifactId>servlet-api</artifactId> <version>2.5</version> <type>jar</type> <scope>provided</scope> </dependency> <dependency> <groupId>javax.servlet.jsp</groupId> <artifactId>jsp-api</artifactId> <version>2.2</version> <type>jar</type> <scope>provided</scope> </dependency> <dependency> <artifactId>el-api</artifactId> <groupId>javax.el</groupId> <version>2.2</version> <scope>provided</scope> </dependency> </dependencies> </project>Create the java launcher With this pom we have all the dependencies we need to run Jetty, Vaadin and Weld together. Lets look at the Jetty Launcher. import javax.naming.InitialContext; import javax.naming.Reference; import org.eclipse.jetty.plus.jndi.Resource; import org.eclipse.jetty.server.Server; import org.eclipse.jetty.webapp.WebAppContext; /** * Simple jetty launcher, which launches the webapplication from the local * resources and reuses the projects classpath. * * @author jos */ public class Launcher { /** run under root context */ private static String contextPath = "/"; /** location where resources should be provided from for VAADIN resources */ private static String resourceBase = "src/main/webapp"; /** port to listen on */ private static int httpPort = 8081; private static String[] __dftConfigurationClasses = { "org.eclipse.jetty.webapp.WebInfConfiguration", "org.eclipse.jetty.webapp.WebXmlConfiguration", "org.eclipse.jetty.webapp.MetaInfConfiguration", "org.eclipse.jetty.webapp.FragmentConfiguration", "org.eclipse.jetty.plus.webapp.EnvConfiguration", "org.eclipse.jetty.webapp.JettyWebXmlConfiguration" } ; /** * Start the server, and keep waiting. */ public static void main(String[] args) throws Exception { System.setProperty("java.naming.factory.url","org.eclipse.jetty.jndi"); System.setProperty("java.naming.factory.initial","org.eclipse.jetty.jndi.InitialContextFactory"); InitialContext ctx = new InitialContext(); ctx.createSubcontext("java:comp"); Server server = new Server(httpPort); WebAppContext webapp = new WebAppContext(); webapp.setConfigurationClasses(__dftConfigurationClasses); webapp.setDescriptor("src/main/webapp/WEB-INF/web.xml"); webapp.setContextPath(contextPath); webapp.setResourceBase(resourceBase); webapp.setClassLoader(Thread.currentThread().getContextClassLoader()); server.setHandler(webapp); server.start(); new Resource("BeanManager", new Reference("javax.enterprise.inject.spi.BeanMnanager", "org.jboss.weld.resources.ManagerObjectFactory", null)); server.join(); } }This code will start a Jetty server that uses the web.xml from the project to launch the Vaadin web-app. Take note that we explicitly use the setConfigurationClasses operation. This is needed to make sure we have a JNDI context we can use to register the Weld beanmanager in. Setup the web.xml Next we’ll look at the web.xml. The one I use in this example is shown next: <?xml version="1.0" encoding="UTF-8"?> <web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" id="WebApp_ID" version="2.5"> <display-name>Vaadin Web Application</display-name> <context-param> <description>Vaadin production mode</description> <param-name>productionMode</param-name> <param-value>false</param-value> </context-param> <servlet> <servlet-name>example</servlet-name> <servlet-class>ServletSpecifiedByTheCDIVaadinPlugin</servlet-class> <init-param> <description>Vaadin application class to start</description> <param-name>application</param-name> <param-value>VaadinApplicationClassName</param-value> </init-param> <init-param> <param-name>widgetset</param-name> <param-value>customwidgetsetnameifyouuseit</param-value> </init-param> </servlet> <servlet-mapping> <servlet-name>example</servlet-name> <url-pattern>/example/*</url-pattern> </servlet-mapping> <welcome-file-list> <welcome-file>index.html</welcome-file> </welcome-file-list> <listener> <listener-class>org.jboss.weld.environment.servlet.Listener</listener-class> </listener> <resource-env-ref> <description>Object factory for the CDI Bean Manager</description> <resource-env-ref-name>BeanManager</resource-env-ref-name> <resource-env-ref-type>javax.enterprise.inject.spi.BeanManager</resource-env-ref-type> </resource-env-ref> </web-app>At the bottom of the web.xml you can see the resource-env we define for Weld and the required listener to make sure Weld is started and our beans are injected. You can also see that we specified a different servlet name instead of the normal Vaadin servlet. For the details on this see the CDI plugin page: https://vaadin.com/directory#addon/cdi-utils The main steps are (taken from that page):Add empty beans.xml -file (CDI marker file) to your project under WEB-INF dir Add cdiutils*.jar to your project under WEB-INF/lib Create your Application class by extending AbstractCdiApplication Extend AbstractCdiApplicationServlet and annotate it with @WebServlet(urlPatterns = “/*”) Deploy to JavaEE/Web profile -compatible container (CDI apps can also be run on servlet containers etc. but some further configuration is required)Add weld placeholder At this point we have all the dependencies, we created a launcher that can be directly used from Eclipse, and we made sure Weld is loaded on startup. We’ve also configured the CDI plugin for Vaadin. At this point we’re almost done. We only need to add empty beans.xml files in the location we want to be included by the beans discovery of Weld. <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/beans_1_0.xsd"> </beans>I had to add these to the src/main/java/META-INF library and to the WEB-INF directory for Weld to pickup all the annotated beans. And that’s it. You can now start the launcher and you should see all kind of Weld and Vaadin logging appearing. Reference: Embedded Jetty, Vaadin and Weld from our JCG partner Jos Dirksen  at the Smart Java blog....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below:
Close