Featured FREE Whitepapers

What's New Here?


Dynamic ADF Train: Adding train stops programmatically

I’m going to show how to add train stops to ADF train programmatically “on-the-fly”. In my use-case I have some ticket-booking application. It has a bounded task flow with train model. At the first stop of the train users input number of passengers and at the following stops they input some passengers’ info. The number of stops with passengers’ info has to be changed dynamically depending on the value submitted at the first train stop. So, the result of described behaviour should look like this:The bounded task flow has the following structure:StartView activity is a page fragment where we input number of passengers and DynamicView activity provides a page fragment to input passenger’s info. At the moment we have only one activity for passenger’s info and I will add extra activities if the number of passengers is greater than one. The inputNumberSpinbox in StartView page fragment submits its value to passengersNumber property of some PageFlowScope backing bean and action for the Submit button is a method of the same bean: public class MainTrain { //Extra added train stops private List<ActivityId> dynamicStops = new ArrayList<ActivityId>(); //Value of inputNumberSpinbox private int passengersNumber = 1; public String buttonPress(){ //The number of extra added train stops is greater than needed if (passengersNumber <= dynamicStops.size()) clearExtraStops(); else //The number of extra added train stops is less than needed if (passengersNumber-1 > dynamicStops.size()) addDynamicStops(); return null; }So, by pressing on Submit button we either add some train stops or clear extra stops depending on the value of inputNumberSpinbox. We save all added dynamic stops in dynamicStops list. Let’s have a look at the clearExtraStops() method: private void clearExtraStops() { for (int i = dynamicStops.size(); i >= passengersNumber; i--) { //Get ActivityId to be removed ActivityId removeActivityId = dynamicStops.get(i-1);//Get current train model and remove train stop TrainModel trainModel = TrainUtils.findCurrentTrainModel(); trainModel.getTrainStops().remove(removeActivityId); //Remove activity from task flow definition getTaskFlowDefinition().getActivities().remove(removeActivityId); dynamicStops.remove(i-1); } }The method removes two things: the train stop from the train model and the activity from the task flow definition. The addDynamicStops() method is going to be much more interesting: private void addDynamicStops() { for (int i = dynamicStops.size(); i < passengersNumber - 1; i++) { //Creating new ActivityId ActivityId activityId = new ActivityId(getTaskFlowId(), new StringBuilder("DynamicView").append(i).toString());//The main trick of the post. //We consider DynamicView activity as a base for new train stop and new activity //Get base activity (DynamicView) and its train stop Activity baseActivity = getBaseDynamicActivity(); TrainStopContainer stopContainer = (TrainStopContainer)baseActivity.getMetadataObject(); TrainStop baseTrainStop = stopContainer.getTrainStop();//Create new Activity based on DynamicView but with new ActivityId ActivityImpl activityImpl = new ActivityImpl(baseActivity, activityId); //Add created activity to the task flow definition getTaskFlowDefinition().getActivities().put(activityId, activityImpl);//Create new train stop based on the DynamicView's train stop TrainStopModel trainStopModel = new TrainStopModel( new TrainStopImpl(baseTrainStop, i+2), activityId); //Add created train stop to the train stop model TrainModel trainModel = TrainUtils.findCurrentTrainModel(); trainModel.getTrainStops().put(activityId, trainStopModel); //Add created activity to our list dynamicStops.add(activityId); } } private Activity getBaseDynamicActivity() { ActivityId baseActivityId = new ActivityId(getTaskFlowId(), "DynamicView"); MetadataService metadataService = MetadataService.getInstance(); return metadataService.getActivity(baseActivityId); }private TaskFlowDefinition getTaskFlowDefinition() { MetadataService metadataService = MetadataService.getInstance(); return metadataService.getTaskFlowDefinition(getTaskFlowId()); }private TaskFlowId getTaskFlowId() { ControllerContext controllerContext = ControllerContext.getInstance(); ViewPortContext currentViewPortCtx = controllerContext.getCurrentViewPort(); TaskFlowContext taskFlowCtx = currentViewPortCtx.getTaskFlowContext(); return taskFlowCtx.getTaskFlowId(); }So, the principal trick of this post is to create new activity and train stops basing on existing ones for DynamicView. In order to implement the idea I created two classes: ActivityImpl and TrainStopImpl. The classes are nothing else than just proxy classes implementing Activity and TrainStop interfaces correspondently. They delegates interface implementation to the base instances except some specific methods like getters for Id and DisplayName: public class TrainStopImpl implements TrainStop { //Base instance private TrainStop baseTrainStop; private int mpassNo; private static final String PASSANGER_FORM = "Passenger's data: "; public TrainStopImpl(TrainStop trainStop, int passNo) { baseTrainStop = trainStop; mpassNo = passNo; }//Specific implementation public String getDisplayName() { return new StringBuilder(PASSANGER_FORM).append(mpassNo).toString(); }public String getOutcome() { return baseTrainStop.getOutcome(); }public String getSequential() { return baseTrainStop.getSequential(); }...public class ActivityImpl implements Activity { private Activity baseActivity; private ActivityId mid; public ActivityImpl(Activity activity, ActivityId id) { baseActivity = activity; mid = id; }//Specific implementation public ActivityId getId() { return mid; }public String getType() { return baseActivity.getType(); }public Object getMetadataObject() { return baseActivity.getMetadataObject(); } ...And one more picture for this post, just to show it’s working:That’s all! You can download sample application for JDeveloper Reference: Dynamic ADF Train. Adding train stops programmatically from our JCG partner Eugene Fedorenko at the ADF Practice blog....

AppSensor – Intrusion Detection

Imagine that you have created a nice web application and secured it to your best. Users came, used it and everything was OK until someone stumbled upon vulnerability in your application and used it. Of course, you analyzed logs and found that the bad guy was looking for the vulnerability for weeks until he found one. Creators of AppSensor intrusion detection framework believe that the above situation should not happen. The application should not just lie there and let itself beat with SQL injections, XSS attacks and whatever else. It should take active measures to protect itself. As the average attacker has to make several attempts to find the vulnerability in the application, it should by possible to detect hacking attempts. The idea of in-application intrusion detection framework seems to be quite original. While intrusion detection frameworks are common in network security world, we found no alternative to AppSensor. If you know about one, please let us know. AppSensor Project AppSensor is part of Open Web Application Security Project (OWASP) project. It started as a conceptual framework and evolved into a real library. The project is currently in beta. AppSensor is located in two places:page on OWASP (Open Web Application Security Project) wiki, google code page.The creator wrote short and easily readable book with project overview, recommendations and best practices. The book is worth reading even if you decide to implement your own solution. If you decide to use AppSensor, getting started guide for developers is available on Google code. Overview High level design is nice and simple. AppSensor has detection points, response actions and intrusion detector. Detection points are able to detect ongoing or probable attacks. They are spread through the application and raise app sensor exceptions whenever intrusion condition is met. For example, “GET When Expecting POST” detection point triggers warning whenever page expecting only POST requests, is requested by HTTP method GET. Or “Cross Site Scripting Attempt” detection point raises an exception if the request contains common XSS attack pattern. Intrusion detector analyses all raised exceptions. If it detects an ongoing attack, it decides which response action is needed. Response action may do anything from raising up log level to shutting down whole application. Which action is selected depends on both suspicious activity and application nature. Where trading application would immediately disable user account, discussion forum would only raise log level and warn user to stop the activity. AppSensor is part of Enterprise Security API (ESAPI) project. Out of the box version supports only ESAPI secured projects. However, our example application is secured with Shiro framework. The integration between both frameworks is described in another blog post. Do no Harm While the whole point of intrusion detection system is to detect and stop an ongoing hack attack, it is equally important not to harm innocent users. AppSensor book suggests to divide detection points into two categories:attack events, suspicious events.If a hidden form variable suddenly contains “‘ OR 1=1″”, there is no doubt that someone is hacking the application. The event is considered an attack event and the system may take strong action (log out, disable account) against the user. On the other hand, unexpected “;'” in request parameter is considered only suspicious. Maybe it was a typo. Maybe it was malicious hacker trying to keep low profile. As we do not know, we can conclude that the system is under attack only if the event occurs multiple times on multiple fields. In this case, the system should record the event and raise log level for current user. Stronger action will be taken only after repeated events. Of course, application context does matter too. Unexpected <img> html tag submitted into trading applications ‘last name’ field must be taken seriously. The same prohibited <img> tag in discussion forum can be taken lightly. The user may be trying to add an animation or whatever to his username. It is annoying, but hardly an attack. Example Application We have been testing AppSensor on SimpleShiroSecuredApplication created as part of Apache Shiro series. You can download example from intrusion_detection branch on Github. Use test class RunTestWait test case to run a web server with application deployed on https://localhost:8443/simpleshirosecuredapplication/ URL. The application has seven users: administrator, friendlyrepairman, unfriendlyrepairman, mathematician, physicien, productsales and servicessales. They all share the same password: ‘heslo’. Add AppSensor First, we have to add AppSensor dependency into the application and enable the framework as intrusion detection provider in ESAPI configuration. We have to do this even if we are not using ESAPI in the project. Add AppSensor dependency to pom.xml: org.owasp.appsensor AppSensor jar compileCreate ‘.esapi’ folder inside src/main/resources folder. Hint for Eclipse users: eclipse package explorer does not show files and folders whose names begin with a . (period). To show them, go to the package explorer view and press little down arrow in the upper right corner of the view. Click Filters and uncheck ‘.* resources’ entry. Unpack AppSensor jar file and copy three configuration files.esapi/appsensor.properties – AppSensor configuration, .esapi/validation.properties – ESAPI input validation configuration, .esapi/ESAPI.properties – ESAPI configurationinto resource/.esapi folder. Alternatively, you can download them from Google code repository. Change ESAPI.IntrusionDetector entry in ESAPI.properties file to use AppSensor as intrusion detector: ESAPI.IntrusionDetector = org.owasp.appsensor.intrusiondetection.AppSensorIntrusionDetectorIntegration Some user actions are considered to be threat only if they are repeated by the same user too many times. Moreover, intrusion detector may decide that ‘log out user’ or ‘disable account’ response actions are needed. All that requires access to underlying application data and APIs. Default AppSensor configuration assumes that application supports ESAPI interfaces. However, SimpleShiroSecuredApplication is secured by Shiro framework which has different set of features. As the integration between two frameworks is more about Shiro than about AppSensor, we moved it to separate post. Detection Points OWASP page hosts extensive list of suggested detection points. It contains almost everything possible. Detection points are divided into several categories:request and session – manipulation with URL, request, cookies, … input – user input contains common XSS attack, unusual character, entered text is longer than input field size, … honey trap – otherwise unused hidden field or request variable with tempting name is_admin is modified, url available only in html comment is requested, … user or system trend – too many log outs across the site, user clicks faster than is humanly possible, user changes his first name field too often, … …A lot of effort went to finding out detection points that could be triggered by an innocent activity. Most of these detection points are not implemented yet. The class AttackDetectorUtils contains the majority of implemented detection points:unexpected request method, xss attack, sql injection, null byte in request parameter, cookies presence, carriage return or line feed presence.Additionally, two servlet filters are available:validate whether user agent changed mid session (UserAgentChangeDetectionFilter), validate whether IP address changed mid session (IpAddressChangeDetectionFilter).Beware: be careful, xss and sql injection detection points need to be reconfigured before use (more about it later). Adding Detection Points Each detection point has unique code. If the application detects an intrusion or suspect event, it can use the code to raise an AppSensorException: if (intrusionDetected()) { new AppSensorException("CODE", "Message to user.", "Log message."); };Note: there is no ‘throw’ clause. Exception constructor automatically triggers it. Alternatively, prepared detection points are available in AttackDetectorUtils class: AttackDetectorUtils.verifyXSSAttack(inputValue)or you can configure servlet filters in web.xml: IpChangedDetectionPoint org.owasp.appsensor.filters.IpAddressChangeDetectionFilter IpChangedDetectionPoint /*Each detection point has intrusion threshold and associated response actions. Threshold is a number of events needed within time period before an associated action is taken. For example, intrusion exception CODE is considered an attack if it is raised four times within ten minutes. If the threshold is reached first time, the event is only logged. User is signed out after second offense. Third offense (e.g. the exception was raised twelve times within ten minutes) will completely disable user account: # number of intrusions in a specified segment of time that constitutes the upper threshold - once crossed, it's considered an "attack" IntrusionDetector.CODE.count=4 # time period (in seconds) IntrusionDetector.CODE.interval=600 # list of actions you want executed in the specified order IntrusionDetector.CODE.actions=log, logout, disableXSS Attack and SQL Injection Detection We start with detection points detecting two popular attacks on web applications: SQL injection and XSS attack. Whenever application reads value from request parameter, the value is passed to detection points: /** * Reads sanitized request parameter value from request. */ protected String getField(HttpServletRequest request, String parameter) { String dirtyValue = request.getParameter(parameter); // detection point: XSS, SQL injection AttackDetectorUtils.verifySQLInjectionAttack(dirtyValue); AttackDetectorUtils.verifyXSSAttack(dirtyValue); return sanitizer.sanitize(dirtyValue); }According to AppSensor book, these detection points detect intrusion whenever an attacker tries to put in SQL injection or XSS attack. If that would be a case, system should take strong action against user. We configured it in appsensor.properties file: # XSS attack -- be careful about these settings # clear attack event, taking action immediately IntrusionDetector.IE1.count=1 # clear log after 20 minutes IntrusionDetector.IE1.interval=1200 # first offense log user out, second disable account IntrusionDetector.IE1.actions=logout, disable# SQL injection -- be careful about these settings # clear attack event, taking action immediately IntrusionDetector.CIE1.count=1 # clear log after 20 minutes IntrusionDetector.CIE1.interval=1200 # first offense log user out, second disable account IntrusionDetector.CIE1.actions=logout, disableHowever, we suggest to be very careful about it. Open ‘personal account page’ of SimpleShiroSecuredApplication and put one of these strings into any field: The dog should fetch the stick as soon as possible.please delete all unused configuration filesAny text with ; in it.SQL injection detection point triggers an exception after any of them. The problem is caused by the word fetch in the first message, by the word delete in the second message and by the symbol ‘;’ in the last message. They are quite common in standard English texts, but all of them raise the SQL injection exception. XSS attack detection point may cause false positives too (however less likely): Hi, thank you for installation scripts and requirements document.cookies has been delicious, please bring them again :)). See you soon, AndyDefault attack patterns are configured in appsensor.properties file. Use standard java regular expressions: # This collection of strings is the XSS attack pattern list xss.attack.patterns=\"><script>,script.*document\\.cookie,<script>,<IMG.*SRC.*=.*script,<iframe>.*</iframe># This collection of strings is the SQL Injection attack pattern list sql.injection.attack.patterns=\\-\\-,\\;,\\/\\*,\\*\\/,\\@\\@,\\@,nchar ,varchar,nvarchar, alter,cursor,delete,drop,exec,fetch,insert,kill,sysobjects,syscolumnsWe strongly suggest to change these lists to something less strict. Response Actions The project contains also list of possible response actions. Suggested actions are divided into following categories:silent – logging change, administrator notification, … passive – show warning, slow down application for the user, … active – log out, disable module, require additional verification (captcha), … intrusive – be careful to stay on the legal sideAppSensor implements only seven response actions. Five are available in our application:log event, logout user, disable user account, send sms to administrator, email admin. disable component, disable component for user.Both disable component response actions lock access to last called URL. They both require additional configuration. Keep in mind, that response action may significantly change code behavior. For example, it may cause seemingly random unexpected user log outs, which may cause unexpected null pointer exceptions or other similar problems. Your design have to be robust enough to handle such situations correctly. Disable Component If you wish to use ‘disable component’ or ‘disable component for user’ response actions, you have to create page that will show instead of blocked page: <title>Access Denied</title> </head> <body> Sorry, you do not have access rights to that area. </body> </html>Only URLs that ends with suffixes configured in appsensor.properties are blockable. We will use only jsp and html pages. All our servlets will have suffix servlet: # This is the list of extensions to check for disabling, ie. jsp (for jsp's), do (for struts 1), UpdateProfile (for the UpdateProfile servlet) disable.component.extensionsToCheck=jsp,html,servlet,It is not possible to configure ‘all pages suffix does not matter’. You have to list all possible suffixes. However, you can configure exceptions that will never be blocked. As we do not use it, we will leave there default configuration: # This is the list of exceptions to disabling components, ie this list should never be disabled disable.component.exceptions=/AppSensorDemo/appsensor_locked.jsp, /AppSensorDemo/login.jsp,/AppSensorDemo/updateProfile.jspEnable disable component actions in web.xml: AppSensorBlockComponent org.owasp.appsensor.filters.AppSensorRequestBlockingFilter redirectURL /simpleshirosecuredapplication/account/accessdenied.jsp AppSensorBlockComponent /*AppSensor redirects blocked requests to redirectURL. Finally, disable component response actions are configured inside detection point configuration in appsensor.properties file. Configuration of ‘disable component for user’ looks like this: # some integer - duration of time to disable IntrusionDetector.ACTION_DOES_NOT_EXISTS.disableComponentForUser.duration=30 # some measure of time, currently supported are s,m,h,d (second, minute, hour, day) IntrusionDetector.ACTION_DOES_NOT_EXISTS.disableComponentForUser.timeScale=mConfiguration of ‘disable component’ is similar. Default duration is 0, so the action does nothing if you do not configure it. Use constant -1 to disable component forever. Custom Response Action If you plan to use AppSensor in your application, you will probably want to add a new response action or modify a default one. Default response actions are handled by DefaultResponseAction class. If you need different set of response actions, you have to implement ResponseAction interface. Following class adds a ‘warn user’ action: public class CustomResponseAction implements ResponseAction { private final ResponseAction delegee=DefaultResponseAction.getInstance();@Override public boolean handleResponse(..., AppSensorIntrusion currentIntrusion) { if ("warn".equals(action)) { Exception securityException = currentIntrusion.getSecurityException(); String localizedMessage = securityException.getLocalizedMessage(); ASUtilities asUtilities = APPSENSOR.asUtilities(); HttpServletRequest request = asUtilities.getCurrentRequest(); request.setAttribute("securityWarning", localizedMessage); return true; } return delegee.handleResponse(action, currentIntrusion); }}Enable new class in appsensor.properties file: # This is the class that handles the response actions AppSensor.responseAction = org.meri.simpleshirosecuredapplication.intrusiondetection.responseaction.CustomResponseActionNote: If you want to extend DefaultResponseAction, you have to override static method getInstance(). The class responsible for initialization would otherwise create an instance of DefaultResponseAction. Intrusion Store All detected intrusions are stored in intrusion store object. Default intrusion store provides very simple implementation: it stores all intrusions in static in memory map. It is not suitable for clustered environment and all stored data are lost after system restart. If you have special needs, you can replace it with own solution. Implement IntrusionStore interface and configure it in appsensor.properties file: # This is the class that handles the intrusion store AppSensor.intrusionStore = org.owasp.appsensor.intrusiondetection.reference.DefaultIntrusionStoreOverall Impressions AppSensor is a great idea and the only intrusion detection framework I know about. The project produced easy to read free e-book and is worth spending some time playing with. Framework design, e.g. division to detection points, intrusion detector and response action are solid and practical. The project is still in beta and it occasionally shows. Default configuration of XSS attack and SQL injection detection points is too strict and we found some small bugs while writing this article. However, project committers response to submitted issues was always fast. Multiple expected small features are missing. It is not possible to say ‘all URLs are blockable’. User has to add all suffixes into extensionsToCheck manually. As default response actions are configured in detection point blocks in appsensor.properties file, we would also expect support for same configuration for custom actions. Or, some classes (DefaultResponseAction) are singletons for no good reason. If you decide to use intrusion detection in your project, your design have to be robust enough to handle all those random log outs and account disables. Reference: AppSensor – Intrusion Detection from our JCG partner Maria Jurcovicova at the This is Stuff blog....

The Perils of Not Unit Testing

Overview Unit testing is a widely accepted practice in most development shops in this day and age, especially with the advent of the tool JUnit. JUnit was so widely effective and used early on that it has been included in the default distribution of eclipse as long as I can remember and I have been programming professionally in Java for about 8 years. However, the drawbacks of not unit testing are concrete and arise acutely from time to time. This article aims to give a few specific examples of the perils of not unit testing. Unit Testing Benefits Unit testing has several basic tangible benefits that has reduced the painstaking troubles of the days when it was not widely used. Without getting into the specifics of the needs and arguments for unit testing, let’s simply highlight the benefits as they are universally accepted by Java development professionals, especially within the Agile community.an automated regression unit test suite has the ability to isolate bugs by unit, as tests focus on a unit and mock out all other dependencies unit tests give feedback to the developer immediately during the test, code, test, code rhythm of development unit tests find defects early in the life cycle. units tests provide a safety net that facilitates necessary refactoring to improve the design of code without breaking existing functionality unit tests, along with a code coverage tool, can produce tangible metrics such as code coverage which is valuable given good quality of tests unit tests provide an executable example of how client code can use the various interfaces of the code base. the code resulting from unit testing is typically more readable and concise, as code which is not so is difficult to unit test. Thus it follows that code which is written in tandem with unit tests tends to be more modular and higher quality.Perils of Not Unit Testing Let’s explore by example how not unit testing can adversely affect code and allow bugs to easily enter a code base. The focus will be on the method level where methods are simple and straight forward, yet there still can be problems when code is not unit tested. Example 1: Reuse some code, but you introduce a bug This example illustrates a situation where a developer has good intentions of reusing some code, but due to a lack of unit testing, the developer unintentionally introduces a bug. If unit tests exists, the developer could have safely refactored and could rely on the unit tests to inform him some requirement had not been covered. Let’s introduce a simple scenario where a clothing store has as system that has users input sales of its clothes. Two objects in the system are: Shirt and ShirtSaleValidator. The ShirtSaleValidator checks the Shirt to see if the sale prices inputted are correct. In this case, a shirt sale price has to be between $0.01 and $15. (Note this example is overly simplified, but still illustrates the benefits of unit testing.) Coder Joe implementes the isShirtSalePriceValid method but writes no unit tests. He follows the requirements correctly. The code is correct. package com.assarconsulting.store.model;public class Shirt {private Double salePrice; private String type; public Shirt() { }public Double getSalePrice() { return salePrice; }public void setSalePrice(Double salePrice) { this.salePrice = salePrice; }public String getType() { return type; }public void setType(String type) { this.type = type; } }package com.assarconsulting.store.validator;import com.assarconsulting.store.model.Shirt; import com.assarconsulting.store.utils.PriceUtility;public class ShirtSaleValidator {public ShirtSaleValidator() { } public boolean isShirtSalePriceValid(Shirt shirt) { if (shirt.getSalePrice() > 0 && shirt.getSalePrice() <= 15.00) { return true; } return false; } }Coder Bob comes along and he is “refactor” minded, he loves the DRY principle and wants to reuse code. During some other requirement he implemented a Range object. He sees its usage in the shirt pricing requirement as well. Note that Bob is not extensively familiar with Joe’s requirement, but familiar enough to feel competent enough to make a change. In addition, their group abides by the Extreme Programming principle of collective ownership. Thus, Bob nobly makes the change to reuse some code. He quickly translates the existing code to use the utility method, and moves on satisfied. package com.assarconsulting.store.validator;import com.assarconsulting.store.model.Shirt; import com.assarconsulting.store.utils.Range;public class ShirtSaleValidator {public ShirtSaleValidator() { } public boolean isShirtSalePriceValid(Shirt shirt) { Range< Double > range = new Range< Double >(new Double(0), new Double(15));if (range.isValueWithinRange(shirt.getSalePrice())) { return true; } return false; } } package com.assarconsulting.store.utils;import java.io.Serializable;public class Range< T extends Comparable> implements Serializable { private T lower; private T upper; public Range(T lower, T upper) { this.lower = lower; this.upper = upper; } public boolean isValueWithinRange(T value) { return lower.compareTo(value) <= 0 && upper.compareTo(value) >= 0; }public T getLower() { return lower; } public T getUpper() { return upper; } }Since there were no unit tests, a bug was created and never caught at time of implementation. This bug will go unnoticed until a developer or user specifically runs manual tests through the UI or some other client. What is the bug? The new code allows 0 to be a price of the Shirt, which is not specified by requirements. This could have been easily caught if there was an existing set of unit tests to regression test this requirement. We could have a minimum set of simple tests that checked the range of prices for a shirt. The set of unit tests could run on each check in of code or each build. For example, the test suit could have asserted the following.$0 = price executes isShirtSalePriceValid to false $0.01 = price executes isShirtSalePriceValid to true $5 = price executes isShirtSalePriceValid to true $15 = price executes isShirtSalePriceValid to true $16 = price executes isShirtSalePriceValid to false $100 = price executes isShirtSalePriceValid to falseIf Bob has these tests to rely on, the first bullet point test would have failed, and he would have caught his bug immediately. Peril – Imagine hundreds of business requirements that are more complicated than this without unit testing. The compounding effect of not unit testing resulting in bugs, repeated code and difficult maintenance could be exponential compared to the safety net and reduced cost unit testing provides. Example 2: Code not unit tested yields untestable code, which leads to unclean, hard to understand code. Let’s continue the clothing store system example, which involves pricing of a shirt object. The business would like to introduce Fall Shirt Sale, which can be described as: For the Fall, a shirt is eligible to be discounted by 20% if it is priced less than $10 and is a Polo brand. The Fall sales last from Sept 1, 2009 till Nov 15th 2009. This functionality will be implemented in the ShirtSaleValidator class by Coder Joe who plans not to write unit tests. Since testing methods is not on his radar, he is not concerned with making the method testable, ie, making short and concise methods to not introduce too much McCabe’s cyclomatic complexity. Increased complexity is difficult to unit test as many test cases are necessary to achieve code coverage. His code is correct, but may turn out something like below. package com.assarconsulting.store.validator;import java.util.Calendar; import java.util.Date; import java.util.GregorianCalendar;import com.assarconsulting.store.model.Shirt; import com.assarconsulting.store.utils.PriceUtility;public class ShirtSaleValidator {private Calendar START_FALL_SALE_AFTER = new GregorianCalendar(2009, Calendar.AUGUST, 31); private Calendar END_FALL_SALE_BEFORE = new GregorianCalendar(2009, Calendar.NOVEMBER, 16); public ShirtSaleValidator() { } public boolean isShirtEligibleForFallSaleNotTestable(Shirt shirt) { Date today = new Date(); if (today.after(START_FALL_SALE_AFTER.getTime()) && today.before(END_FALL_SALE_BEFORE.getTime())) { if (shirt.getSalePrice() > 0 && shirt.getSalePrice() <= 10 ) { if (shirt.getType().equals("Polo")) { return true; } } } return false; } }The problems with this code are numerous, including misplacement of logic according to OO principles and lack of Enums. However, putting these other concerns aside, let’s focus on the readability of this this method. It is hard to ascertain the meaning of this code by just looking at it in a short amount of time. A developer has to study the code to figure out what requirements it is addressed. This is not optimal. Now’s lets think about the testability of this method. If anyone was to test Joe’s code, after he decided to leave it this way due to his NOT unit testing, it would be very difficult to test. The code contains 3 nested if statements where 2 of them have ‘ands’ and they all net result in many paths through the code. The inputs to this test would be a nightmare. I view this type of code as a consequence of not following TDD, i.e. writing code without the intention of testing it. A more TDD oriented way of writing this code would be as follows. package com.assarconsulting.store.validator;import java.util.Calendar; import java.util.Date; import java.util.GregorianCalendar;import com.assarconsulting.store.model.Shirt; import com.assarconsulting.store.utils.PriceUtility;public class ShirtSaleValidator {private Calendar START_FALL_SALE_AFTER = new GregorianCalendar(2009, Calendar.AUGUST, 31); private Calendar END_FALL_SALE_BEFORE = new GregorianCalendar(2009, Calendar.NOVEMBER, 16); public ShirtSaleValidator() { } public boolean isShirtEligibleForFallSale(Shirt shirt) { return isFallSaleInSession() && isShirtLessThanTen(shirt) && isShirtPolo(shirt);}protected boolean isFallSaleInSession() { Date today = new Date(); return today.after(START_FALL_SALE_AFTER.getTime()) && today.before(END_FALL_SALE_BEFORE.getTime()); } protected boolean isShirtLessThanTen(Shirt shirt) { return shirt.getSalePrice() > 0 && shirt.getSalePrice() <= 10; } protected boolean isShirtPolo(Shirt shirt) { return shirt.getType().equals("Polo"); } }From this code we can see that the method isShirtEligibleForFallSale() reads much like the requirement. The methods that compose it are readable. The requirements are broken up amongst the methods. We can test each component of the requirement separately with 2-3 test methods each. The code is clean and with a set of unit tests, there is proof of its correctness and a safety net for refactoring. Peril – Writing code without the intention of testing can result in badly structured code as well as difficult to maintain code. Conclusion The above examples are only simple illustrations of the drawbacks of foregoing unit testing. The summation and compounding effect of the perils of not unit testing can make development difficult and costly to a system. I hope the illustrations above communicate the importance of unit testing code. Source Code peril-not-unit-testing.zip Reference: GWT and HTML5 Canvas Demo from our JCG partner Nirav Assar at the Assar Java Consulting blog....

Application Performance and Antipatterns

Any application you pick up, there are some issues – big or small. There will be copy-paste code, mistakes, algorithms which could have better thought through. But what distinguishes an antipattern from these normal errors is that like patterns these antipatterns are recurring throughout the code base. In my recent experience in dealing with performance issues, I had observed certain recurrent themes that are undermining the overall application performance. Most of these antipatterns are well documented but it seems we do not learn from others mistakes. We need to make our own mistakes. I am recounting some of the common patterns that I observed in the recent months.Excessive Layering – Most of the underlying performance starts with the excessive layering antipattern. The application design has grown over the usage of controllers, commands and facades. In order to decouple each layer, the designers are adding facades at each of the tiers. Now, for every request at the web tier, the request call goes through multiple layers just to fetch the results. Imagine doing this for thousands of requests coming in and the load the JVM need to handle to process these requests. The number of objects that get created and destroyed when making these calls add to the memory overhead. This further limits the amount of requests that can be handled by each server node. Based on the size of the application, deployment model, the number of user’s, appropriate decision need to be taken to reduce the number of layers. E.g. if the entire application gets deployed in the same container, there is no need to create multiple layers of process beans, service beans(business beans), data access objects etc. Similarly, when developing an internet scale application, large number of layers start adding overheads to the request processing. Remember, large number of layers means large number of classes which effectively start impacting the overall application maintainability.Round Tripping- With the advent of ORM mappings, Session/DAO objects, the programmer starts making calls to beans for every data. This leading to excessive calls between the layers. Another side issue is the number of method calls each layer start having to support this model. Worse case is, when the beans are web service based. Client tier making multiple web service calls within a single user request have a direct impact on the application performance. To reduce the round tripping, the application needs to handle or combine multiple requests at the business tier.Overstuffed Session- Session object is a feature provided by the JEE container to track user session during the web site visit. The application start with the promise of putting very minimal information in the session but over a period of time, the session object keeps on growing. Too much of data or wrong kind of data is stuffed into the session object. Large data objects will mean that the objects placed in the session will linger on till the session object is destroyed. This impacts the number of user’s that can be served by the application server node. Further, I have seen, application using session clustering to support availability requirements but adding significant overheads to the network traffic and ability of application to handle higher number of users. To unstuff the session object, take an inventory of what all goes there, see what is necessary, what objects can be defaulted to request scope. For others, remove the objects from session when their usage is over.Golden Hammer (Everything is a Service) – With the advent of SOA, there is tendency to expose the business services, which can be orchestrated into process services. In the older applications, one can observe similar pattern being implemented with EJBs. This pattern coupled with the bottom up design approach at times, means exposing each and every data entity as a business service. This kind of design might be working correctly functionally, but from the performance and maintenance point of view, it soon becomes a night mare. Every web service call adds overhead in terms of data serialization and deserialization. At times, the data(XML) being passed with web service calls is also huge leading to performance issues. The usage of services or ejb’s should to be evaluated from application usage perspective. Attention needs to be paid on the contract design.Chatty Services – Another pattern observed is the way the service is implemented via multiple web service calls each of which is communicating a small piece of data. This results in explosion of web services and which leads to degradation of performance and unmaintainable code. Also, from the deployment perspective, the application starts running into problems. I have come across projects which have hundred plus services all getting crammed into a single deployment unit. When the application comes up, the base heap requirement is already in 2Gb range leaving not much space for application to run. If the application is having too many fine grained services, then it an indication towards the application of this antipattern.The above mentioned antipatterns are frequent causes of application performance issues. The teams usually start with the right intentions but over a period of time, things will start slipping. Some of the common reasonsLack of design standards and reviews processes – even if these exists, the delivery pressure is leading to skipping these processes Team members inexperience or narrow view leads to every programmer only looking at their module and nobody is looking at the overall application performance Continuous Integration(CI) tools not integrated with compliance check tools like PMD, Checkstyle, FindBugs etc No focus on profiling of the application on regular basis during the code construction phase Not evaluating the results from the Load tests to decipher and fix the underlying issue (blaming the poor infrastructure setup)What are the other antipatterns you have observed that have contributed to the degradation in the application performance. Do share! Reference: Application Performance and Antipatterns from our JCG partner Munish K Gupta at the Tech Spot blog....

Lean IT fundamentals & principles

The roots of Lean IT are – as the name suggests – in Lean Management. The basic principles of Lean Management had been developed in Toyota’s Production System, that was built back in the 40s of last century. Lean IT is really nothing new, but as an application or adaptation of the basic principles of Lean Management to software development and maintenance the topic not commonly known. The two key elements of Lean IT are the avoidance of waste and continuous improvement process. [1, 2, 3] Lean Transformation  The introduction of lean in a company is also known as Lean Transformation Program. Because it is a big organisational and cultural change, Lean IT is not only the introduction of a single methodology. A Lean Transformation requires actions in several dimensions. The approach of McKinsey considers the following example (4 +1) dimensions: [4]Mindset and Behaviour, Process Efficiency, Performance Management, Organisation & Skills und Voice of the Customer.For each of these dimensions, there is a number of tools, methods and approaches. As part of a Lean Transformation, these are introduced and performed in each single unit/team of the company. The transformation of a working unit/team usually takes between 3-4 months. Afterwards, a continuous improvement process will be setup, to keep the status quo and even get further improvements. Lean IT Principles Steven C. Bell describes in his book ‘Lean IT, Enabling and Sustaining Your Lean Transformation’ following nine principles and/or elements of Lean IT: [5]Respect for People, Pursuit of Purpose, Constancy of Purpose, Proactive Behavior, Quality at the Source, Voice of the Customer, Flow/Pull/JiT, System Thinking und Culture.The following picture shows some of the most important dependencies and relations.Relationship with Agile Development Methods Agile software development methodologies (e.g. SCRUM, XP) and Lean IT are very similar and this is no coincidence, because many of the agile methods and paradigms have their roots in Lean Management. There are some important differences:The popular agile development approaches are highly tailored to self-organizing team. It can work great, if the team members are suitable for self organisation. Lean IT is not based on a self-organizing teams, this means that under Lean IT also classic leadership structures work well. Agile methods are usually limited to insulated working teams and/or composites oft co-located teams. Lean IT is more of an enterprise approach. This is a great advantage, but there is a risk that Lean IT fails for the entire company. Lean IT can be used outside of the actual software development and software maintenance. This is especially true for infrastructure units.Despite all the differences, both approaches have in common is that they try to (i) increase productivity and customer orientation (ii) by means of a continuous improvement process (iii) to achieve increased customer benefits.   Sources and further ReadingsToyota-Produktionssystem; Wikipedia;(Stand2. August2011);http://de.wikipedia.org/wiki/Toyota-Produktionssystem Lean Management; Wikipedia;(Stand2. August2011);http://de.wikipedia.org/wiki/Lean_Management Lean IT; Wikipedia;(Stand2. August2011);http://en.wikipedia.org/wiki/Lean_IT The time is right for lean in payments; (2009); http://www.mckinsey.com/clientservice/Financial_Services/Knowledge_Highlights /Recent_Reports/~/media/Reports/Financial_Services/Lean_In_Payments.ashx Steven C. Bell and Michael A. Orzen; Lean IT, Enabling and Sustaining Your Lean Transformation; Productivity Press; (2010); ISBN-13: 978-1439817568Reference: Lean IT fundamentals & principles. from our JCG partner Markus Sprunck at the Software Engineering Candies blog....

Top 10 Things Every Software Engineer Should Know

Please take a second to look at our brand new Java Resource Collection. The following top ten list collects some important things I have learned in the last eighteen years as IT professional. It is a very personal selection and doesn’t necessarily reflect the opinion of a software engineering organisation. There is no strict ranking in the list – though I tried to put the more important things to the top.The technical and business know-how is more important for young software engineers and the soft skills getting increasingly relevant for senior software engineers. 1) Fundamentals of Emotional Intelligence Almost all of us work with a lot of people. In my first year after university, I had the opportunity to work on a clear big task without any customer and the need to talk a lot with peers. It was pure haven! Just do a complex task and have fun with the compiler. Later the trouble started with more complex tasks, increasing responsibilities and the need to work with people I didn’t liked at all. During my professional life, I attended some so called soft skill courses. In these lesions I learned a lot about communication techniques, negotiation strategies and team dynamics. All this have been mechanical tools or psychological theories. Good to know, but the concept of Emotional Intelligence is something different. The Wikipedia definition of Emotional intelligence starts with the sentence “Emotional intelligence (EI) is the ability to identify, assess, and control the emotions of oneself, of others, and of groups.” [1] The important key word in this sentence is emotions. Emotional intelligencedescribes the role of emotions in our lives. Some years ago, I attended a project meeting with some senior management and the boss of my boss said something to me which sounded like “Hey Markus, you forgot to give me the information XYZ in time!”. I felt embarrassed, like a culprit and explained him that he was not right. The result was that I won the discussion with him and form that day I lost an important supporter in the company. My reaction was stupid and worthless. Yes, I won one battle, but lost the war. The root cause of this disaster was an automatic reaction on my site and a reciprocal effect between this senior management guy and me. With better sens of self and self-regulation, I would have been able to manage the situation in a better way. If you leave sometimes a meeting and say to your self “Oh shit! Why did I say this?”, maybe it would be a good idea to learn something about Emotional Intelligence and yourself. My favorite breakdown of emotional intelligence is:Intrapersonal intelligence describes the ability to have positive relationships and/or good communication between people. This means that you understand what people fell and need. The key competences of intrapersonal intelligence are:Sense of Self: – recognize the own feelings, emotions and reactions – mindfulness to get a better awarenessSelf-regulation – controlling the current inner state – bring own automatic reactions to mind and interruptPersonal leadership – know and lead the parts of your own personalty – care about own strengths and weaknessesInterpersonal intelligence describes the introspective and self-reflective capacities. Know your self, your emotions and what your weaknesses or strengths are, being able to control your own reactions. The key competences of interpersonal intelligence are:Empathy – recognize the feelings and emotions of others – express sympathy in anappropriate wayReduction of Automatic Reciprocal Effects – bring automatic reactionswith othersto mind and interrupt (if needed) Creating Relationships – create mid- and long-term relations with others2) Understand the Business of your Customer How can you design and implement good software without deep understanding of the purpose or use? The answer is easy: “If you don’t know the WHAT, you can’t decide about the HOW.” A deep understanding of your customer’s and/or users’sbusinesswill lead tobetter requirements, designs, implementations and tests.Most of the software’s functionality creates no business value. The challenge is to select the functionality which creates business value. The better you know the business the higher is the probability to implement the best system. 3) Minimum One Programming Language for each Mainstream Development Paradigm The discussion what is the best programming language has a religious character, it’s more a question of belief. I don’t like to preach my personal belief about the bestlanguageshere, but one thing is important: “Learn more programming languages, at least one for each mainstream development paradigms.”procedural programming languages (C, COBOL, PL/I, FORTAN, etc.) object-oriented programming languages (Smalltalk, Java, C++, etc.) functional programming languages (Erlang, Clojure, F#, etc.) declarative programming languages(SQL, XSLT, regular expressions, etc.)Its a good idea to know at least one multi-paradigm programming languages like Python, Java, C++ or C#. You find many listsof programming languages by type or other categoriesin the web [2]. Dependent of your industry, personal preferences and daily tasks you should select your individual top 1o list of programming languages. Learn them and try to use at least 3 of them on a regular base. The old saying “If your only tool is a hammer, all your problems will look like nails” is particularly true for development paradigms. 4) Know your Tools There is a huge number of tools specializing indifferent disciplines like: requirements management, software & database design, software configuration management, build & deploy, continuous integration, development, debugging, profiling, code analysis or testing. It should be mentioned that specialist from infrastructure/operations have also toolboxes with interesting capabilities, e.g. network monitoring, network analysis, operation system analytics, penetration testing, log file analysis, database performance tuning. A software engineer can’t know all tools in detail, but he/she should know the key concepts and underlying technologies. Knowing the right tool and how to use can increase the productivity and quality.Spend some time to learn about tools. 5) Standard Data Structures, Algorithms and Big-O-Notation When I stated to develop software it was absolutely necessary to know a lot about data structures and algorithms. The reason for that was the missing availability of standard implementations. Today most languages have comprehensive libraries for container, sorting and other operations. Still it makes sense to know more. There are two main reasons:correct use of the standard libraries and some times you need individual solutions.You should be able to analyse your own or others code. TheBig-O-Notation is the standard method to describe the expected consumption of time or memory depending from the number of data. [3] If a manual analysis is to difficult, just make a micro benchmark and measure with test data of different size. Draw it in a plot and find a good fit of a possible model function. This is always better than nothing. 6) Don’t Trust Code without Adequate Test Ten years ago, I trusted my code. Why not? After 8 years C++ with excellent skills and a lot of experiences. I just coded, tested and everything was working well. But over the years I made and saw a lot of errors. Because of these errors, I lost the trust in my own and others code. Today, I don’t trust code until it passed:unit test, integration & system tests, checks of performance and memory with real world data, static code analysis, measure code coverage of test, load & stress tests and peer review.This sounds over engineered, but you have to spend the time either during development or during maintenance. I favor to do the work once with good quality and not to spend my time withtroubleshooting. 7) Basics of Project Management, Lean Management and Agile Concepts Even you don’t like to work as a project manager, you work in teams and at least have to organize your own work. To get along with technical leads you should understand their wording and way of thinking. Today everybody can work as project manager, scrum master or technical lead. Spend time to learn about management, because sometimes you should manage these guys. A good example is effort estimation. My personal experiences say, that if you ask a software engineer about the effort of a task you get in 80% of the cases a dramatic underestimation of the effort. A software engineer tends to estimate just the good case without unexpected problems. This causes delays and/or poor quality because quite often the unexpected problems just happen.An other problem is the Definition of Done. The project manger means everything is done and often the developer estimates just the technical stuff. Last week I had such a case. The developer estimated just one week of work. And after a complete planning, we saw several months effort. The developer estimated the time for implementation and forgot to estimate documentation, security concept, data protection issues, alignment with workers councils, reviews, project management efforts, deployment, etc. 8) Key Metrics of Software Development Know what happens in your software, process, team and your own work. It is very difficult to control something what you can’t count. I encourage you to have question and try to find a real world measure as answer. Then you can have target values, do your work and find out if it worked out. Important is the word “real world measure”. In software engineering we find a lot of obscure measures and/or derived metrics. E.g. the so calledmaintainability index (MI) [4]: MI = 171 – 5.2 x ln(avgHV) – 0.23 x avgCC(g‘) – 16.2 x ln (avgLOC) + 50 x sin (sqrt(2.4 x perCM)) where HV is the Halstead Volume, CC is the Cyclomatic Complexity, LOC is the lines of code and perCM is the percentage of comment lines. This is not what I call a real world measure and I don’t understand this. My advice is easy: “Never use a measure and/or metric you don’t understand 100%. Some times it is enough to take some glasnuggets and count them (see also How Lean IT helps to reduce waste due to interruptions in software development?).” 9) The Root Cause of the Last Defect Maybe your last error was not as severe, but to learn more about the root cause and negative effects   you should analyse it.What was the root cause? In what development phase came the error in the software? How could it be detected earlier? Would a tool help to avoid it? Would a rule help to avoid it? Was it a qualification problem? Is the working environment (lot of interruptions, etc.) the root cause? Is it an documentation problem? Or maybe a communication problem? What are the costs to fix it? Are in the affected component more errors? Are the test cases good/complete enough?You see a lot of question and the list is still not complete. The most important point is, to find the root cause to get better over the time. This works for your own qualification and way of working. And it works for your team. You just have to ask some question. 10) Understand the Infrastructure I spend the my first 10 years in IT without thinking more than a minute about infrastructure. It was not necessary, because I didn’t work in an enterprise environment. At the moment I work for a bank (sorry for these Lehman Brothers stocks, nobody asked me). In a bank you have a lot of these infrastructure people. They are really different form software engineers. But, I don’t like to discuss here the differences and possibilities to get along with them. Important is their language. Infrastructure peoples talk in “Information Technology Infrastructure Library (ITIL)”. Spend at least some days to learn this ITIL terminolgy. [5]Some terms are completely different uses as developers do. The second important thing is, that in infrastructurethe people are much more specialized than developers. Sometimes a developer has just one question and needs five infrastructure guys for the answer. The ITIL stuff is maybe the glue between the people ininfrastructure. Further Readinghttp://en.wikipedia.org/wiki/Emotional_intelligence  http://en.wikipedia.org/wiki/Categorical_list_of_programming_languages http://www.leepoint.net/notes-java/algorithms/big-oh/bigoh.html http://www.virtualmachinery.com/sidebar4.htm  http://en.wikipedia.org/wiki/Information_Technology_Infrastructure_LibraryReference: Top 10 Things Every Software Engineer Should Know from our JCG partner Markus Sprunck at the Software Engineering Candies blog....

Application Security at Scale

This week’s SANS AppSec conference in Las Vegas took on Application Security at Scale: how can we scale application security programs and technologies to big organizations, to small organizations and across organizations to millions of programmers world wide. You can find the presentation slides here. Lots of hilights for me: The conference was kicked off by Jeremiah Grossman from WhiteHat Security who made it clear that the problem of web application security alone is much bigger than we can take care of with the people and technology that we have today. We need to try different things like:Game-ification: get developers interested and involved in Appsec using games and challenges like capture-the-flag, or the Elevation of Privilege card game (a game I have to try out) Use peer pressure and score cards between teams, products, business units – drive better application security through competition (as we learned later, Cisco is one of the organizations score carding business units and products to drive improvement in software security programs) Good, simple (and I will add inexpensive) online training to get as many developers as possible up to speed on secure design and coding Write good, usable security frameworks and libraries and build security in by default into the major application frameworks – unfortunately we don’t know what frameworks will get widely adopted until they are widely adopted, so we will always be playing catch-up Build security into the developer’s workflow – this is what SD Elements is doing Use WAFs and virtual patching where it makes sense – to raise the bar on attacks by plugging simple issues found by scanners (WAFs used properly could block more than 2/3 of web application vulnerabilities, the kinds that scanners find); and to secure legacy code that nobody wants to try to figure out and fix by hand or in shops where it is too expensive and slow to get fixes out (in many Agile / Devops shops, it’s faster to fix and deploy the code than it is to put in a patch to the WAF). Bug Bounty programs – if this works for Google and Facebook, it could work for you.Chris Eng at Veracode presented some metrics collected from the scans that they have done for customers over the past 18 months. The interesting thing for me was the correlation between attack data (from the Verizon 2011 Data Breach Report)and vulnerability data, the intersections highlighting what we need to focus on. Just like last year (and the year before) SQL injection is the leading problem: 32% of apps scanned had SQL injection vulnerabilities, 20% of attacks are SQL injection. The XSS Problem (and some Solutions) According to Veracode’s data, 68% of web apps have XSS vulnerabilities. This is no surprise after you listen to Jim Manico explain in detail what programmers have to do to prevent XSS. Even getting every developer building web apps to understand all of the different rules for context-correct output encoding and escaping isn’t going to solve the problem: there are too many details for developers to take care of without missing something or making mistakes. “It’s more complex to stop XSS in large-scale apps than it is to do applied crypto and key management properly…. We have never seen a web app that can’t be attacked through XSS”. But there is hope – in a later presentation he explained how Context-Aware Auto-Escaping (aka Auto-Encoding) technology like JXT (a close-to-drop-in replacement for JSP, if you are writing well-formed JSP) and Ivan Ristic’s work on Apache Velocity Auto-Escaping can help protect at least some web apps from XSS. The most promising new technology for me was HTML5 iFrame Sandboxing which looks like it could actually be dropped in today to protect apps, at least if your customers are using modern browsers. I also learned about JavaScript object freezing and sealing to help protect rich client apps. I was on a panel that looked at application security in small companies, together with Nick Galbreath at Etsy and Cameron Morris at Partnet – a small company that builds online web shopping portals for the US DoD. At Partnet everyone owns security: many of the developers have been trained on application security, all of them understand the OWASP Top 10, all code that is checked in is reviewed. I presented a case study on our AppSec program, what worked and what didn’t for us, from startup to now. Nick built on an earlier presentation by Zane Lackey at Etsy which explained some of the security controls in Etsy’s frameworks and Continuous Deployment pipeline, and the extensive monitoring and instrumentation feedback loops that they have from production back into development. This includes cool automated checks on changes to high-risk code (they have automated tests that hash specific pieces of code, if the hash value changes the build system automatically alerts the AppSec team and provides them the change set for code review). Being able to deploy several times a day means that they can deploy new code (including security fixes) extremely quickly with high confidence. Nick also presented on rate limiting to monitor and control event activity in production. I am critical of Continuous Deployment – too many people who try to follow this model put speed ahead of reliability and security, and unnecessarily put their customers at risk. They don’t know when they have crossed the line from a web startup to running a real business. But if you are going to try this and want to do it right, learn everything that you can from Etsy. Mobile AppSec – Android is a Train-Wreck From the panel on mobile Appsec: Google has a good page on writing secure apps for Android (I am guessing it is this page) but it’s clear that most developers don’t know about it. The consensus was that Apple’s IOS is the most secure smart phone platform, and Android is a train wreck – according to one of the researchers (Georgia Weidman at Bulb Security), 1/2 of the default Android apps have serious security vulnerabilities. Secure Frameworks and APIs Chenxi Wang’s Day 2 keynote emphasised the importance of isolating security code and sharing and reusing security code through APIs. This was reinforced in a later panel by Jason Chan at Netflix and Adam Migus at E*Trade – like us, both these organizations rely on simple, extensible secure frameworks or APIs that developers can use to take care of problems like identity management, permissioning, crypto, secure transport, validation. At E*Trade, they find that most security vulnerabilities are because developers didn’t use this code (or didn’t use it properly). It doesn’t cost that much to write and support a secure framework – a few smart people can take care of this for the rest of the organization. And there are Open Source examples today like Apache Shiro that we can use to solve a lot of common security problems. Pen Testing An excellent panel on “inside the mind of a pen tester”, how expert pen testers think and work and approach problems, the tools that they use (I learned about chaining multiple attack proxies like Burp Suite and Zap together to take advantage of the different strengths of each tool). The most important thing in the pen test is for developers and management to get a clear understanding of the risks in the application: what kind of problems the testers found, how serious were they, what you have to do to fix them and what you have to do to prove that you fixed them. The real value of pen testing, like any other kind of testing, is the information that you get out of the test. If you’re not learning from pen tests, if the next time the tester comes back and tests the same system and finds the same problems, what are you paying for? These pen testing experts had mixed opinions of WAFs – if a customer has a WAF it is usually installed out-of-the-box without tuning, and doesn’t present more than a speed bump to a determined attacker. But like anti-virus protection, it will stop most drive-by attacks – some big sites are seeing that as much as 10% or even 20% of their traffic is potentially dangerous, and a good WAF should be able to catch at least some of this. Most memorable quotes from the conference There is no such thing as an internal application. Jeremiah Grossman, WhiteHat Security You can checkbox compliance but you can’t checkbox security. Monica Bush, University of Wisconsin-Madison The closing message was sobering. We need more people who understand AppSec and who can write secure code – a lot more people. Big companies may only have a few AppSec generalists supporting thousands of developers, most companies don’t have anyone at all. This isn’t enough. Point-in-time assessments like pen tests aren’t enough either, because the attack space is always changing and the code is always changing – in a few weeks or months at most the results of a pen test may be invalidated. What we are doing today isn’t enough and it’s not going to scale. We need more security burned in and we need continuous security testing, which means more people who understand AppSec and better and more effective tools. Reference: Application Security at Scale from our JCG partner Jim Bird at the Building Real Software blog....

REST endpoint for integration using Apache Camel

REST is an architectural style used for organizing resources and when applied to HTTP-based services allows building stateless, decoupled, scalable services. HTTP methods, HTTP headers, and mime-types all allow a developer to achieve the REST style. Frameworks like Jersey and Fuse Services Framework (Apache CXF) can be used to speed up the development and deployment of services trying to achieve a RESTful style, and in this blog post I’d like to discuss how to build the backend of a resource that relies on integration provided by Fuse Mediation Router also known as Apache Camel. Just as an aside, a link that I’ve had tucked away in the recesses of my bookmarks may be of interest for those of you wondering whether your architecture is indeed RESTful or just the same highly coupled RPC style that REST tries to alleviate. Roy Fielding, who wrote his dissertation on REST, actively asserts the notion that hyerlinks within resource representations are a must for REST styles, and even further clarifies the uncertainties around implementing REST. The source code for this sample can be found on my github repository Fuse Mediation Router is FuseSource’s enterprise-grade, hardened version of Apache Camel that provides a comfortable DSL for describing integrations, mediations, and routing. It’s free, open-source, and has an Apache License. For those unfamiliar with Mediation Router/Camel, take a look at an introduction from Jon Anstey (co-author of Camel in Action)at DZone’s Enterprise Integration Zone: Apache Camel: Integration Nirvana. We will be using Mediation Router to help write a simple integration between a REST endpoint and a resource files on a file system. I’ll be using camel-cxfrs component to expose the REST endpoint and will be using the camel-file component to read a directory on the file system. The intention of the sample is to describe the configuration necessary to expose the REST interface with Mediation Router, integrate with a backend somehow, transform the data into an appropriate REST response, and send back the response. To get started, let’s focus on how to set up the REST endpoint. To do so, you would create a JAX-RS resource that describes the java methods that will act as REST endpoints. This sample code requires familiarity with Java API for RESTful Web Services aka JAX-RS. For those unfamiliar, here are some great tutorials to follow along that help to understand JAX-RS. @Path("/customerservice/") public class CustomerServiceResource {// NOTE: The instance member variables will not be available to the // Camel Exchange. They must be used as method parameters for them to // be made available @Context private UriInfo uriInfo;public CustomerServiceResource() { }@GET @Path("/customers/{id}/") @Produces("text/xml") public Customer getCustomer(@PathParam("id") String id) { return null; }@PUT @Path("/customers/") public Response updateCustomer(Customer customer) { return null; }}As you can see, the annotations are the JAX-RS annotations that describe the operations, HTTP methods, and mime-types involved with the REST endpoint. Notice, the return values are all null as this class will not actually be used to handle the requests that come in to the endpoint; the Mediation Router routes will be responsible for processing and responding. Note, however, that instance members are not available to the Mediation Router exchanges, i.e., any instance members injected via the JAX-RS @Context annotations will not be available. To make them available, add them as parameters to your methods. Declaring the CXF-RS endpoint with Mediation Router can be done one of two ways: Directly in the endpoint configuration like this: from("cxfrs://http://localhost:9090/route?resourceClasses=com.fusesource.samples.CustomerServiceResource")Creating it directly in the configuration requires less xml configuration but offers limited flexibility. Another option is creating a separate bean that’s responsible for the endpoint and then referencing it within the endpoint configuration: from("cxfrs:bean:rsServer")The bean rsServer should be defined in the camel context. An example: > <cxf:rsServer id="rsServer" address="http://localhost:9090/route" serviceClass="com.fusesource.samples.CustomerServiceResource"/>This approach allows you to decouple the endpoint configuration and allows to be quicker and less verbose in the endpoint configuration. Both options are shown in the sample code, although the first option is used. That’s all the configuration required to expose the REST endpoint with Mediation Router. Fairly simple. The next step is to consume a file from the file system based on what comes in from the REST endpoint. The contents of the file will be returned to the client of the REST call. To do this, we use the camel-file component and enrich the Exchange with a pollEnrich call in the DSL: .setHeader(Exchange.FILE_NAME, simple("test-${body}.xml")) .pollEnrich("file:src/data?noop=true", 1000, new CustomerEnricher())We cannot use any dynamic expressions in the pollEnrich call, so we set a header that the file component understands before we do the enrichment. In this case, the body of the REST message is an identifier that can be used to template the file-system resource. Lastly, we can attach some additional processing to the route: .process(new CustomerServiceProcessor())The intent of the example, as described above, is to show how to configure the endpoint and attach it to further Mediation Router processing. Note, the Message Exchange Pattern (MEP) for the REST endpoint is InOut and expects a response. The example is not meant to be a complete end-to-end solution as that will vary depending on intended functionality. Please note above the links to Roy’s discussions on what REST is and is not. If I have left something out, or you need more clarification around the example, drop me a comment and we can discuss. Reference: REST endpoint for integration using Apache Camel from our JCG partner Christian Posta at the Christian Posta Software blog....

GWT and HTML5 Canvas Demo

This is my first experiment with GWT and HTML5 Canvas. My first attempt is to create rectangles, with just a few lines of code I came up something like this:Code:public class GwtHtml5 implements EntryPoint { static final String canvasHolderId = "canvasholder"; static final String unsupportedBrowser = "Your browser does not support the HTML5 Canvas"; static final int height = 400; static final int width = 500; final CssColor colorRed = CssColor.make("red"); final CssColor colorGreen = CssColor.make("green"); final CssColor colorBlue = CssColor.make("blue"); Canvas canvas; Context2d context; public void onModuleLoad() { canvas = Canvas.createIfSupported(); if (canvas == null) { RootPanel.get(canvasHolderId).add(new Label(unsupportedBrowser)); return; } createCanvas(); } private void createCanvas(){ canvas.setWidth(width + "px"); canvas.setHeight(height + "px"); canvas.setCoordinateSpaceWidth(width); canvas.setCoordinateSpaceHeight(height); RootPanel.get(canvasHolderId).add(canvas); context = canvas.getContext2d(); context.beginPath(); context.setFillStyle(colorRed); context.fillRect(100, 50, 100, 100); context.setFillStyle(colorGreen); context.fillRect(200, 150, 100, 100); context.setFillStyle(colorBlue); context.fillRect(300, 250, 100, 100); context.closePath(); } }And my Spring balls experiment with some codes that I found on the Web.Reference: GWT and HTML5 Canvas Demo from our JCG partner Mark Anro Silva at the GlyphSoft blog....

Spring 3 and Java EE 6 – An unfair and incomplete comparison

The first draft of this small article had the title ‘Spring & Java EE – Comparing Apples and Oranges’. During writing this, I learnt that it is possible to compare Spring Framework and Java EE, but it is always an unfair and incomplete work. The evolution of Java for Enterprise and Spring Framework are strongly connected to each other. Both leaned from each other and partly copied good concepts. In Table 1 you can see a simplified timeline with some key milestones of Java Platform for Enterprise and Spring Framework. This table explains, that it makes only sense to compare Java EE v6 with Spring v3.0. Earlier versions of Spring Framework and J2EE are outdated and Java EE v7 is still not published (actually delayed to Q2 2013). Table 1: Timeline of Java Platforms and Spring FrameworkYear Java Platform, Standard Edition Java Platform, Enterprise Edition Spring Framework Key Milestones2000 J2SE v1.3J2EE v1.2.1EJB 2 – difficult deployment descriptors – difficult testing – lot of redundantartifacts2001J2SE v1.320022003 J2SE v1.4 J2EE v1.42004Spring v1.0 First Spring Framework – dependency injection (IoP) – no Java EE application server -competitor to J2EE2005Spring v1.22006 J2SE v5 Java EE v5 Spring v2.0 Java EE 5 (EJB 3.0) – a lot of functions inspired by Spring – elimination of component, home and remote interfaces2007Spring v2.520082009 Java SE v6 Java EE v6 Spring v3.0 Java EE 6 (EJB 3.1) – interfaces are optional – singleton beans – cron-like scheduling – embeddable containers Spring 3.0 – Spring Expression Language – MVC-Framework with improved REST support20102011 Java SE v 7Spring v 3.1So, in table 2 the main building blocks of Spring Framework are listed. The Inversion of Control Container is the core function of Spring Framework. To understand how this works you may also check A minimal Java application based on Spring framework(sample code uses Spring 2.5.6.A, but the main principles are the same for Spring 3.0). Table 2: Selected Building Blocks of the Spring v3.0 FrameworkInversion of Control Container - Inversion of Control – Dependency Injection – Configuration with XML files and/or annotations (auto wiring)Model-View-Controller Framework - Domain objects (Models) -Usually JSP templates (Views) -DispatcherServlet as front-controller for ControllerAspect-Oriented Programming Framework - Basic AOP for cross-cutting concerns in aspects – Interception based and is configured at runtime – Configuration with XML files and/or annotationsBatch Framework -Processing large volumes of records or tasks.Including: logging, tracing, transactions, job management, resource management, exception convertingData Access Framework - Support is provided for popular frameworks – JDBC, iBatis, Hibernate, JDO, JPA, Oracle TopLink, Apache OJB, and Apache CayenneTransaction Management Framework - Abstraction mechanism (JTA only supports nested transactions and global transactions, and requires an application server)Remote Access Framework - Working with various RPC-based technologies available on the Java platform both for client connectivity and exporting objects on serversSource & Further Reading: http://en.wikipedia.org/wiki/Spring_Framework In table 3 the main standards and components of the Java Platform, Enterprise Edition 6 are listed. This table makes clear that Java EE 6 contains of a lot of standards and is not just a framework. Table 3: Selected Building Blocks of the Java EE 6 ArchitectureSource of Picture The Java EE 6Tutorial,p39 Client Machine – Java Server Faces (JSF 2.0) – JSP Standard Tag Library (JSTL) – Ajax with JavaServer Faces Technology – Facelets (XHTML) – Support for the Expression Language (EL) – Templating for components and pages – Java Servlet Technology (Servlet 3.0) – Internationalizing and Localizing Web Applications Java EE Server – Enterprise JavaBeans (enterprise bean) components – JAX-RS RESTful web services – JAX-WS web service endpoints – Enterprise Beans as POJO (Session and Message-driven) – Managed Beans as POJO – Interceptors – Dependency Injection for Java (JSR 330) – Contexts and Dependency Injection, CDI (JSR 299) – Annotations to minimize deployment descriptor needs – Asynchronous method invocation in session beans – Declarative and programmatic security – Java Message Service (JMS) API – Java Persistence API entities Persistence – Java Database Connectivity API (JDBC) – Java Persistence API 2.0 – Java EE Connector Architecture – Java Transaction API (JTA)Source & Further Reading: http://olex.openlogic.com/wazi/2010/get-started-with-jee6/ Practical Experiences When I started to learn Spring Framework 3.0 some years ago, I implemented a small web application with a lot of Spring functions. To be honest, more than necessary for this task. This small application had extensive test code for unit and integration test automation. It has 100% line and branch coverage. Some time later, I decided to implement exactly the same application based on Java EE 6 Architecture to compare the two approaches.Both technologies worked well, have almost the same code size and a good maintainability. One significant difference is the support of testing. Spring Framework has an excellent support for testing and Java EE 6 has here some weaknesses. For Java EE 6 you can use Embedded Glassfish, but this approach is annoying slow (long start-up time for embedded container) and touchy in the configuration. A further outstanding feature of Spring Framework is the easy integration of legacy applications. It is easier to renew an old application in a step by step approach, e.g. with JDBC, transaction management and small parts of IoC in the beginning. ConclusionForm the architectural point of view the models of Spring and Java EE are competitors. Depending on the strategy and constraints of your organization both show strengths and weaknesses. A combination of both can’t generally be recommended, because it is either waste of the EJB Container functionality and/or difficult to maintain. In some selected cases, it may make sense to use Building Blocks of Spring Framework in a Java EE 6 application, e.g. Spring Batch, Data Access. Spring Framework is not a standard – it is a product and an implementation of a framework. This means, there is no alternative vendor. Please, keep this always in mind.Reference: Lightweight Java Enterprise Architectures – An Unfair and Incomplete Comparison Between Spring 3 Framework and Java EE 6 from our JCG partner Markus Sprunck at the Software Engineering Candies blog....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below: