My First Java Project

My first Java project
Java Programming. Downloaded from Googled, labelled as free to reuse, under Wikipedia Commons. URL: http://commons.wikimedia.org/wiki/File:Java_Programming_Cover.jpg

It’s been a while since I decided to use Java as my new programming language of choice. Since my last post I have been honing my Java skills, with my first java project.

I had a project I needed to do that is about analyzing academic papers, comparing them in various ways, sorting them, writing out reports etc.

My first instinct was to use PHP (which I learnt a few years ago – and I really like the language) but the only development environment I have is on a server. I could set up a development environment on my desktop, but it did not seem worth it for the times that I would use that environment in anger.

The other option was C++, but that went against what I was trying to do for this project, as you’ll understand if you read earlier posts in this project.

So, the obvious solution was to use Java.

As my first Java project, it seemed to have suitable complexity for a first time Java user, but allowing for the fact that I have a programming background. I suspect that the project would not be suitable for a complete newbie to programming.

The details of what the project has to deliver is not that important but the lessons learnt are important, as these are things that I can use in later, larger, projects. These included:

  • Reading in a CSV (Comma Separated Variable) file. This is always a useful thing to be able to do. It’s one of those fall backs that is useful to have in your armoury as it makes accessing files such as spreadsheets and databases easy to do as they invariably have an option to save as a CSV file. Of course, it’s usually possible to access spreadsheets and databases directly (which is what I used to do under C++ using ODBC) but for this project I thought I would get to grips with CSV first. After some searching, I came across a package called CsvReader. It’s not the most basic package (which is a good thing), and it did have good reviews. I had some challenges installing it, but that was not to do with the package but the fact that this is the first time I have installed a Java package. Once I had sorted that out, it worked perfectly.
  • Writing out a latex/PDF file. One of the things I wanted to do was produce a half decent looking report. My initial thought was to write out a text file and then use Word to manually format it. This, for many reasons, is not a good idea; not least of all as it would be labour intensive every time I produced a new version of the report. I like to think that I had a flash of inspiration (but perhaps it is the obvious thing to do) and I decided to write out a latex file that I would convert to PDF via a suitable editor (my editor of choice at the moment is TexStudio, although I have used WinEdt in the past). This seems to work pretty well and I can now produce nice looking reports, without having to worry too much about the look of the final document as Latex will handle this as long as I have some idea of the structure as I develop the program. Of course, the beauty is that the report is complete, once the program has been run, without any need for any other processing/formatting. In a future blog I’ll provide a few more details as to how I did this as a) I think it’s interesting and b) I am sure that there are better ways of doing things and I’d like to get some ideas for developing the system further.
  • Writing files. An obvious thing I had to do when writing out latex files, was to learn out to write out files. As any Java programmer will know, this is very easy using the PrintWriter package.
  • Sorting arrays. I had a need to sort an array on one of the fields. In fact (see below), this involved sorting an array of class instantiations. This was probably the most difficult thing I did when developing my first Java project, and it took a while before it came together. But following a few examples from the web, and I had this working pretty quickly. It certainly seems easier than C++, which always seemed complicated and involved having to have friends of classes. I’m sure that there are easier ways in C++ but I never really got my head around it.
  • Classes and data. This is not really a Java thing and maybe I am totally wrong, but I quickly found that my data and member functions were making the class quite large, so I decided to have a class called (say) ‘Papers’ and another class called ‘PapersData’. The PapersData class simply holds all the data and the Papers class provides access to it, as well as providing all the other functionality. This leads to (at least in the way I do it) too many getters and setters, but it does separate the data/functions. But, the main reason I did this is because I wanted to hold different data types for my various objects and an ArrayList (or other array type objects) would not allow this. I am happy to be corrected but I was trying to recreate the struct (i.e. a class) type concept of C++. Anyhow, it seemed to work for what I wanted, but whether it would scale to larger projects is another matter.

The system I have ended up with, seems to work well. Whether it is scalable to larger projects remains to be seen, but it has certainly been a very good learning experience. I have doubts that if I was trying to learn C++, even with a good programmimg background, I would have progressed as fast as I have with Java.

No doubt, other people would pick up C++ faster than I would have done but, for me, Java is a lot easier to learn.

The other big bonus is the Eclipse IDE.  No doubt, I have only scratched the surface but the autocomplete (Ctrl/0) and the suggested error correction (Ctrl/1) are my new best friends!

So, as my first Java project, I think, has been a worth while exercise and I have learnt a great deal.

 

Videos on the basics of Java

Videos on the basics of Java

Whilst looking through youtube (see my previous posts – here and here – for why I am doing this), I have come across some very nice youtube videos on the basics of Java. A very nice series of videos by Jose Vidal starts from the the basics of Java:

Eclipse Java ‘Hello World’ Introduction Tutorial

… but moves quickly on to topics such as:

Javadoc

Constructors

Java Wrapper Classes

Java Immutable Classes

Java Packages

Java Arrays

Java Expanding Array

Java Selection Sort

Java Multidimensional Array

Java Inheritance Tutorial

Java Polymorphism

Java Reading a CSV file

Java Reading and Writing Binary Files

Java Serializable interface: Reading and writing Objects to a file

Java Arraylist

Java HashSet HashMap Demonstration

Java Generics Tutorial

Java Iterators

Java Building your own Iterator using Inner Classes

Eclipse Tips and Tricks for Beginners Tutorial

Java Set: HashSet TreeSet LinkedHashSet

Jose has many other videos in his collection and if you find the above useful you should check out his youtube channel.

 

Other useful videos

As I have said before, there are many (many, many) youtube videos out there that can help teach the basics of Java. They are too numerous to list, but here are just a few that I have either found helpful, or that I plan to watch later (and I might just keep adding to this list so that all the videos I found useful are kept in one place).

In no particular order.

Iterators Part 1 (Java)

I realise that many of the topics above are very basic for a seasoned programmer but I think that there is also some advanced material there as well (e.g. inheritance, polymorphism, iterators etc.), so hopefully it will be of interest to a wide variety of people.

I can certainly say (from my own persepctive) that to get your head around this in MFC/VS/C++ would certainly be a lot mor work (for me anyway).

The football prediction project

You can read more about this project by looking at the posts for this football prediction project.

Blackjack in Java

Whilst looking through youtube (see my previous posts – here and here – for why I am doing this), I came across a very nice resource that showed how to implement Blackjack in Java. I am sure the video’s presenter will be the first to admit that the project is far from complete (for example, there is no double down, no splitting etc.). But that is not the point of the exercise. It was to show how to develop a complete application rather than developing a complete Blackjack player in Java.

The video was very useful. It started from an empty project and added classes as they were needed (Card, Deck, Player) and showed how they all linked together.

To be honest, it did not teach me anything I did not know already about the language syntax (as it is very close to C++) but it was very good to see somebody build a complete application from scratch.

Blackjack in Java

If you want to watch the video, take a look at the video below.

Previous interest in blackjack

Blackjack in Java
Blackjack Table (Downloaded from Google, via Wikepedia Commons – labelled as free to reuse); URL: http://commons.wikimedia.org/wiki/File:Blackjack_board.JPG

I must admit to being a little biased towards this video, as I have published an academic paper on blackjack, which evolved a blackjack player so that it could play blackjack and win money. Of course, we could never use this in a casino (for many reasons!) but it was a fun project and it would be interesting to ressurect the project in Java – but that really is for another day. But I might go and see if I can find some open source classes for a deck of cards, then I could do my version of Blackjack in Java!

The football prediction project

You can read more about this project by looking at the posts for this football prediction project.

Learning Java, the first steps

In this post, I describe my first steps in learning Java. Not that I am a newbie to programming. I have been doing it for years, but mostly in C++, or various mainframe languages and editors. But I recently made the decisio to change to Java.

Learning Java

Learning Java
Java Programming. Downloaded from Googled, labelled as free to reuse, under Wikipedia Commons. URL: http://commons.wikimedia.org/wiki/File:Java_Programming_Cover.jpg

I thought it was about time to start learning Java. At the moment, I am based in Malaysia so I cannot simply reach to my bookshelves (in my office in Nottingham) and get one of the many Java books I have accumulated over the years. Of course, I could go to the University library (and may well do so) but at the moment, all I have is Google (other search engines are available!).

There are obviously lots of internet forums around, but these can be quite difficult to follow, can digress quite a lot and are often in the form of Q&As which might not be the best way to learn.

Youtube for Java

I often overlook Youtube, but as I found out in my previous post, there are some fantastic resources out there.

I was fortunate enough to come across a series of videos that introduce various topics. I was drawn to it as the first episode was called “Episode 1: Hello Java! Getting Started with Eclipse.”

I found this quite informative. I think you have to have some programming experience to keep up with it, and there is quite a lot of digressing – but I still found it a good watch.

In fact, I have watched the first 15 videos. Some are very useful, some not so. Tutorial 8 was not easy to follow (it’s about Networking). In my mind this was advanced stuff, but the presenter, I think, has a networking background so, I suspect, to him, it was pretty basic.

But overall, this series was worth watching, and I will watch some of the later episodes in the future.

Others are available

I was attracted to the videos above as it used Eclipse. But looking around Youtube, there are many (many, many) alternatives. Some are more advanced, some confusing, some too simple, some incorrect (or not well explained) – and that is from my limited knowledge.

Understandable?

I’m pleased to report that I understood most things. Of course, knowing C++ helps a lot, as the syntax is pretty similar. It’s also good that I don’t have to worry about whether I am using MFC (Microsoft Foundation Classes), whether, once I have console application, I could ever convert it to a Windows application, what to do when I get binding errors etc. As far as I can see, once you have your imports (for packages and calsses) sorted, then everything should be okay.

There also seem to be some nice features, such as ArrayLists, the Scanner class and the Collections object (which has sorting apparently – saves me having write a qsort and sort out friend functions etc.).

So ….

As far as learning Java, I think that I have found some excellent resources on Youtube and I would urge anybody learning Java to tap into this resource. Indeed, for many techical problems, Youtube seems a great resource.

Questions

Of course, I have questions. What is a call to super? What does implement do? How should I structure packages? How should I structure my classes and the GUI (it seems to me that it would be easy to write a console application and then plug on a GUI? – not something I’d want to try with Visual Studio).

But all these questions seem pretty easy to answer, with access to the right resource, but only time will tell.

The football prediction project

You can read more about this project by looking at the posts for this football prediction project.

Swing or SWT when using WindowBuilder

In my last post I was trying to decide whether to use Swing or SWT when using WindowBuilder. I am using the Eclipse IDE.

Learning to use Swing or SWT

The problem is, there does not seem to be any clear cut view as to which is best. This post on stackexchange is typical.

Researching the options

Most of the forums/tutorials I found on Google tended to jump in at the deep end, assuming that you know the basics and it was not very helpful in making this decision. Then I came across this youtube video.

The sound quality is not that good, but it takes you from how to install WindowBuilder, all the way to using Swing to create a window. I actually followed the tutorial in real time and managed to get the same results (a window with a text button in it), which I though was quite impressive.

The tutorial uses Swing, and I have since heard that Swing is better for Windows applications. Whether that is true or not, I don’t know, but as I have to make a decision, I have decided (for now anyway) to adopt Swing.

If I am honest, I don’t think it actually makes that much difference. The applications that I intend to write will be fairly simple from a GUI point of view, so I doubt that I would really push the limits of either Swing or SWT.

Decisions made

Swing or SWT for the Eclipse IDE?
Eclipse IDE (Dowsnloaded from Google, vis Creative Commons – 06 Mar 2014)

Whether, this is a good decision remains to be seen – but at least I am making some decisions. I have decided to use Eclipse, and now Swing.

I think that these represent two of the major decisions and even they are not reversible. I can always switch at a later date.

Probably of more importance now, is to actually write some Java code. Knowing C++ should make this relatively easy, but bolting everything together within a GUI might be the major challenge!

 

 

 

What Java GUI development tool shoud I use?

In my previous post, I outlined the reasons why I was switching to Java as the programming language of choice for a football prediction system that I am developing. In this post, I try to decide what Java GUI development tool I should use? Should I use Swing or SWT as my Java GUI builder?

Getting to grips with Java

Eclipse IDE (Dowsnloaded from Google, vis Creative Commons - 06 Mar 2014)
Eclipse IDE (Dowsnloaded from Google, vis Creative Commons – 06 Mar 2014)

I still need to get my head around Java. As I said before, I have done some Java programming but I’ll need to get a lot more skilled with the language. But, as one of the key reasons for switching to Java is to try and find an easier way (for me) to develop a graphical user interface, I thought that I ought to try and make that decision now.

WindowBuilder

Digging around, there are some tools that I could use. Eclipse WindowBuilder seems to get a decent write up, so I decided to use that. At the time of writing, I have installed this as part of my Eclipse installation. So far, so good.

Swing or SWT as the Java GUI helper?

Once I started digging a little deeper, I found that I had to make another decsion, whether to use Swing or SWT as my Java GUI builder. This does not seem so easy to answer. Searching for comparisons (this one is typical) tends to brings up pros and cons for each, with the conclusion that there is no right or wrong answer, in the same way that there is not clear cut answer whether you should use Eclipse, NetBeans or IntelliJ as your IDE.

So, at the moment, I am still none the wiser whether to use Swing or SWT. Unless anybody has any insights (please feel free to leave a comment)  I think I might have to try both. Probably eaiser said than done, as I am no where near the stage where I can get a Window to display on the screen, let alone decide the best tool to do it!

I’ll try to report more later.

Time to switch to Java for a football prediction project

Time to switch to Java for a football prediction project
Programming Books (Clive Darra, Creative Commons)

I have decided that it was time to switch to Java for a football prediction project that I have been planning for some time.

I want to do the project justice so I thought I would start from the most basic decision. What programming language should I use?

The Problem with C++

For the past 20 years, I have been using C++ (before that I was using all sorts of mainframe languages). I can do what I need to do with C++, but I have never been entirely happy with it.

As a language, I quite like it, but it is all the stuff that goes with it that has always been frustrating. And most of that stuff is around Visual Studio (VS), MFC (Microsoft Foundation Classes) and Templates. Don’t get me wrong, this is not a Microsoft bashing exercise. I know of many people who use VS and are very happy with it, develop great applications and know how to use the tools that are available.

But I have never really got my head around it. I run into all sorts of problems with namespaces, linking errors, deciding whether to use MFC (Micosoft Foundation Classes), or not – and then regretting it, whether to use the classes available in VS so that I could port my code to another compiler should I wish to do so; and the list goes on.

The end result is that I have never really developed as a C++ programmer. I can do all the usual stuff (classes, inheritance, operator overloading, polymorphism) but I have never been able to get to grips with Windows (Graphical User Interface – GUI) programming and so have always stuck with a Command Line Interface (CLI).

To be honest, using a CLI has served me well and I have managed to churn out some nice programs. But for my football prediction project, I really want to develop some sort of GUI. The question I asked is, should I have another go at getting to grips with MFC, Windows programming and all that is need to get an application developed in VS to display a Window on the screen, or do I change to something else, in the hope that I will find it a little easier to understand?

So why Java?

After a lot of soul searching, and Googling, I decided that it was time to switch to Java. I have written a couple of programs in Java before, but nothing too much beyond “Hello World”.

But is like C (syntax wise), and like C++ (class wise), it is well supported, many of my academic colleagues use it, it is platform independent and, I hope, that GUI programming is a little easier than VS.

More questions that answers

Of course, making the decision to use Java just raises a whole load more questions. What IDE should I use (I have chosen Eclipse), how big is the learning curve, can I easily access Excel files, can I (should I) use MySQL, what Java tools are available to support developing a a GUI etc.

All these questions will have to wait. For now, I have to learn the basics of Java and get my head around the Eclipse IDE (Intergrated Development Environment).

Wish me luck.

 

Football fixture forecasting. Are you any good?

Chelsea have produced artist's impressions of how a new stadium at Battersea Power Station would look. Photograph: Chelsea/PA (19 Jun 2013: downloaded via Google where the image was labeled as free to reuse)
Chelsea have produced artist’s impressions of how a new stadium at Battersea Power Station would look. Photograph: Chelsea/PA (19 Jun 2013: downloaded via Google where the image was labeled as free to reuse)

Football fixture forecasting is something I have expressed an interest in recently. Actually, this blog contains a forecasting category, that you might be interested in.

In my last post, I mentioned a crowdfunding project that I am trying to get off the ground. This prroject aims to investigate football fixture forecasting, utilising methodologies such as Artificial Neural Networks and algorithms based on Darwin’s prnciples of natural evolution (survival of the fittest).

Doing research for this project, I looked around a few football forums and found that there is a brisk business in football forecasting competitions (usually just for fun).

For example:

  • FootballForums.net have a Football Forum thread, which can be accessed here.
  • Total Football Forum also has a prediction thread.

I mention these now, as they might be of interest as the new season approaches.

Of course, one of football’s most well known pundits (Mark Lawrenson) makes predictions every week (on Radio Five Live I believe) and these are tracked at various web sites including MyFootballFacts.com. This site does a great job of showing the actual league tables and what would have happended if all Lawro’s predictions had come true. It’s interesting to see the fairly signisficant differences league table based on actual resuts. I guess it is no that surprising that there are large differences as, if Lawro, was an excellent preditor all the time, the bookies would be bankrupt.

I think I am also right in saying that Lawro tries to predict the actual score. This has got to be difficult. If I ever get my project off the ground, I think I’ll just try to predict whether the home team will Win, Lose or Draw.

 

Using ELO ratings for match result prediction in association football

I occasionally comment on a scientific paper that is of interest to me. This time, it was:

Lars Magnus Hvattuma and Halvard Arntzenb (20010) Using ELO ratings for match result prediction in association football, International Journal of Forecasting, 26(3), 460-470 (doi).

This paper falls into the broader categories of Football, Forecasting and Sport, which I have blogged about in various ways.

ELO ratings were originally designed for Chess. In brief, if you play an opponent who is lot weaker than you and you lose, your rating would reduce a lot more than if you were beaten by a player who is only slightly weaker than you. Similarly, if you beat a player who is a lot stronger than you, your rating will increase a lot more than if you beat a player who is only slightly stronger than you. It is usually (if not always) a zero sum game so that what ever points you gain (or lose) the same number of points is lost (or won) by your opponent. There are a number of variations as to how ratings are updated but once you have a rating you can compare yourself to another player (or team) to decide who is the better player.

The idea behind ELO ratings have now been used for a number of sports. Relevant to this blog post, it has been applied to football. In fact, there is a web site that maintains a list of ELO ratings for world football. At the time of writing, the latest match played (on the 15th Aug 2012) was a friendly between Albania and Moldova. The score was 0-0 and the teams gained 5 points (Moldova) and lost 5 points (Albania) respectively. Their points are now 1461 (Albania) and 1376 (Moldova). The fact that Albania were higher ranked meant they lost points against a lower ranked team when they could only manage a draw. They also had home advantage, though I am not sure that this is a factor in this model. Looking at the web page, I don’t think so.

In the paper, which is the subject of this post, the authors define two ELO models to try and rate teams. The first uses a basic ELO to work out ratings and then uses the difference between the two ratings as a covariate (a variable that could be a predictor of the study under investigation) in a regression model. This seems reasonable, as the the ELO rating is a measure of the current strength of the team. A second ELO model is also used which also takes into account the margin of victory. That is, a 3-0 win is better than a 2-1 win.

To compare the two ELO models the authors define another six prediction methods. The first (which they call UNI) simply chooses an outcome at random. The second (FRQ) uses the frequency of home wins, draws and away wins to make its prediction. For example, over a range of matches if two thirds result in a home win, then FRQ would predict that two thirds of the matches it predicts are also predicted as home wins. It is noted that UNI and FRQ are quite naive prediction methods. Two other benchmarks are derived from a paper by John Goddard (Goddard J. (2005) Regression models for forecasting goals and match results in association football, International Journal of Forecasting, 21(2), 331-340 (doi)). The fifth comparative model is based on bookmakers odds, by taking the average of a set of odds from various bookmakers. The final model is almost the same as the fifth, but instead of using the average odds, the maximum odds are used.

The paper also uses various staking strategies. A fixed bet of one unit, a stake in proportion to the odds, to return one unit and stakes based on the Kelly system which is an optimal staking system if you know the exact probabilities.

The various prediction models and staking plans were tested on eight seasons of data (2000-2001 to 2007-2008). Previous seasons data was used, to train the various models.

So, what were the results? The two ELO models were found to be significantly worse than two of the other six models (those based on bookmaker odds), but the ELO models were better than the other four models. The conclusion is that this is a promising result but perhaps more covariates are required to predict even more accurately.

 

Odds-setters as forecasters: The case of English Football

I sometimes comment on a scientific paper that has caught my eye. This time, it was (comments on other papers are also available):

David Forrest, John Goddard and Robert Simmons (2005) Odds-setters as forecasters: The case of English football, International Journal of Forecasting, 21(3), 551-564 (doi).

This paper falls into the broader categories of Football, Forecasting and Sport, which I have blogged about in various ways.

The premise of this paper is that the odds set by bookmakers are in fact predictions by experts. Not a bad assumption to make given that the odds that are set equates to real money coming in, or going out. It was also motivated by the fact that a previous study had concluded that a statistical model performed better than newspaper forecasters.

But is it true that bookmakers odds are good predictors?

To answer this question, the authors took almost 10,000 games, between 1998 and 2003, and compared the bookmaker’s predictions with forecasts made by a statistical model. The model they use is fairly complex (in my opinion) although understandable (and, importantly, reproducible) to the interested, and mathematically competent,  reader The model also looks back over fifteen seasons so if you wanted to implement it, there is quite a lot of data that needs to be collected.

The comparisons are quite interesting between the different forecasting methods. Over the five seasons considered, 1998-1999 (1944 matches), 1999-2000 (1946 matches), 2000-2001 (1946 matches), 2001-2002 (1946 matches) and 2002-2003 (1945 matches) it showed that the statistical model performed better in the earlier seasons, but by the end of the period under investigation the odds-setters were performing better. Looking at just the odds-setters they did worse in the two seasons at the ends of the period under investigation (that is, 1998-1999 and 2002-2003). This is presumed to be there being more noise in the match results.

If you look at the data another way, there is little to choose from the home, draw and away predictions between the two prediction methods.

A follow up comparison attempts to estimate if the odds-setters are capturing all the information contained in the model (if you look at the paper, the model contains many elements such as whether a team is still in the FA Cup, if the match is important to a particular team, attendance information etc.) All the odds-setters have is one figure (the odds) and that has to incorporate everything in order to provide a prediction. The figures presented suggest that, over time, the odds-setters get better at incorporating information into their odds. It is shown that incorporating the bookmakers odds into the statistical model gives superior performance, suggesting that the bookmakers are privy to information that is not included in the benchmark model. This was not the case in the earlier paper, where this test failed when considering tipsters.

Next the paper investigates if indiscriminate betting (placing bets,  using identical stakes, on any outcome) on matches produces better or worse returns than if you follow the benchmark model. Indiscriminate betting, maybe not surprisingly, produced a loss at a rate that a bookmaker might expect in the long term (termed the over-round), around 12%. By comparison if you place a bet on the outcome with the highest expected return, as indicated by the benchmark model, then you would do much better. Over the five seasons you would have only lost -0.2%.

A possible downside is next considered. The benchmark model is the same for every season, whereas the odds-setters are (probably) learning each season or, perhaps, using different people/methods to set the odds. Actually, the model did not give superior predictions by refining it as the simulation progressed from season to season, or even as the season progressed.

One of the conclusions of the paper is that experts (in this case odds-setters) are actually quite good at predicting the outcome of matches. This is attributed to a variety of factors including the fact that their livelihood depends on it and also the sector is much more competitive (due to technology and tax reforms) that have required them to become better to remain competitive.

My conclusion? Use the model, and incorporate bookmakers odds – that gives you the best predictions. But it is quite a lot of data to collect. Alternatively, develop your own predictions methodology and see how it performs.