Just spent the day fooling around with Octave. It's basically a consol based terminal that is equipped with all the basic functions of Matlab with additional C++ assignments and operator overloading (+=, ++, etc...) and some of Python's libraries (mainly concerning mathematics). Also some basic functions used to study control systems (like bode and nyquist plots) are enabled without installing additional packages.

With additional packages like image-1.0.13.tar.gz and odepkg-0.6.10.tar.gz, some additional applications can be performed like image processing and solving ordinary diffrential equations. Unfortunately, there is no support for advanced GUI building (there is the Zenity package that builds a very basic GUI interface).

In addition to the basic consol based Octave interface, there is a GUI based one called QtOctave. Some functions may actually give better results with QtOctave in comparison with consol-based Octave like reading RGB images and plotting functions in 3D (mesh and contour). Besides, the GUI platform highly resembles that of Matlab (being it having a visible Terminal with an Editor space to import directly an .m file and a window to show the Commands' History). Eventhough the latest version of QtOctave is built upon Octave 3.0.0 while the latest version of Octave is 3.2.4, this can be solved by simply installing Octave 3.2.4 and changing the configuration of QtOctave as such:

Restart QtOctave in order to configure the libraries of Octave 3.2.4.

An important note on how to exit Octave 3.2.4: type 'quit' on the command line.

Another important note: don't forget to change the directory name to where the .m file you're working on exists. This can be done by either going File -> Change Directory or typing chdir('~/Desktop/Filename') where the folder where the .m file and other data sources, like auxiliary functions and imported images, lie is called "Filename".

Also typing "clc" on the command line doesn't clear the terminal on QtOctave, but using the eraser like icon on the Menu bar.

Octave 3.2.4 can be downloaded here.
QtOctave 0.7.2 can be downloaded here (based on Octave 3.0.0).
(The latter software can also be downloaded from the main sites, but I found these two executables as the best versions that mimick Matlab).
Additional packages can be downloaded at Source Forge, the home-site of Octave.

More info about Octave:

http://en.wikipedia.org/wiki/GNU_Octave
http://www.gnu.org/software/octave/index.html
http://en.wikipedia.org/wiki/List_of_numerical_libraries
http://en.wikipedia.org/wiki/List_of_statistical_packages

Programming Tutorials:
Basics of Octave Programming
Programming Differences between MATLAB and Octave
Basically any MATLAB tutorial available (pay attention to required packages which should be installed in order to execute some Matlab functions that are imported from additional toolkits like image processing) besides ones dealing with GUI applications.
Special thanks to arithma for mentioning Octave during our discussion here.