While reading Kevan Stannard's article on ColdFusion 9 object creation improvements post I was curious as too how Railo would stand up on the exact same test. The results are interesting:
( Seconds to create 100k objects and Objects created per second )
ColdFusion 8:
ColdFusion 9:
Railo 3.1.0.22:
OpenBD:
I could not achieve the figures Kevan achived perhaps an environment difference?
My setup is ColdFusion 8 and 9 are 64 bit on Windows 7 dev editions with 512mb ram assign in CFadmin. Railo was just Railo express. This is on my main work machine ( Dell XPS 1530 4g ram Windows 7 64bit ).
Now the interesting thing to note is that CF8 and 9 took well over 500mb of ram in task manager and Railo express was less then 100mb.
I'm not knocking CF9 as we haven't had any big issues even with 8 with our applications but I thought the results were interesting enough to share.
And yes we will be upgrading to CF9 and I can't wait!
PS someone with OpenBD should run this perhaps later tonight I'll install the express version and run the same test.
** Update I used the stable version on OpenBD ready to run on the same system and got excellent results.
PPS Do not laugh at me using IE it just had the least amount of addins and I used it so the screen shots would be cleaner.
Code:
test.cfc
speed.cfm
Flex and Coldfusion developer.
Server administration.
Links
Blog Archive
-
►
2010
(38)
-
►
August
(9)
- ColdFusion and cfquerynew and date times - timesta...
- ColdFusion Lists, SQL IN and cfqueryparam
- iPhone and Flash - Sucess
- ColdFusion jrun and the JVM
- ColdFusion and CFReport
- jQuery or EXTjs
- iPhone 3Gs and Flash player
- Easy way to embed ColdFusion code into your blog
- Flex and the MATE framework in Melbourne
-
►
August
(9)
Reccomended:
Search
27 comments:
Interesting result Paul, thanks for sharing!
I'm a bit confused on what the numbers represent. Is that seconds, followed by instantiations?
@Ben yes seconds to complete the task then objects per second.
Interesting for sure. Thanks!
Thank you Paul.
Awesome! That's a huge performance!!!
BTW I also am running JDK 1.6.14 for ColdFusion 8
Nice...
Is it because CF9 is still based on the CF8 (and 7 and 6) code base and it is getting inefficient compare to Railo with new architecture?
My guess is Railo is much faster here because it is running on jetty and CF9 is running on the internal server?
My tests (deployed as a war on Sun GlassFish ES 2.1) show CF9 being about twice as fast as Railo 3.1.0.22.
CF9:
2.233 :seconds to complete
44782.8034035 :objects per second
My Railo tests resulted in similar times as the ones you posted.
Also I just checked openbd and it cannot render the test as is (fails on < and i++).
So the jee server cause slowness? I would like to test CF8, 9 Railo and OpenDB all on the save platform perhaps Tomcat and see how they compare.
Your CF9 tests are very impressive implying we should all switch to Solaris?
I am running this test in a local Dev environment on a Dell Studio XPS w/ Core i7 920 and 6GB on Vista Home Premium 64bit.
I am a big fan of GlassFish and v3 is even faster. I am sure in deployment, on a real server you could get the test times down to 1 second.
Interesting results. With your CF9 test did you run the test just once or multiple times? I found that the first time I ran the test it created around 10,000 objects. I ran the test a couple more times and each time the number of objects increased up to a limit of around 25,000 per second.
I ran it a bunch of times it did get upto ~15k but no higher with 512 ram I adjusted the jvm memory to 2048 and it got as high as 17.5k but that was the peak for me.
The CF9 test I ran was inside an XP virtual machine with nothing but CF9 installed. It was using the standard CF9 config (Java version 1.6.0_12, 512MB heap) with debug turned off. Just tried again now and was getting around 30K objects per second. I am just curious because you have a much better spec machine than I am testing on so I would have expected higher CF9 numbers (considering your CF8 test was significantly faster than mine). I'll look forward to seeing your OpenBD test.
I look forward to seeing the OpenBD results as well. Thanks for posting your finds. =)
good good
Paul,
Would it be possible to share the code that you used to compare the performance?
Rupesh
cf9 is not faster than cf8, the only differents between this versions are that cf9 has 2 new caching mechanism:
- Cache Template In Request
- Component cache
when you disable this caches in admin, cf9 has the same speed as cf8.
@Markus Moser
Markus you are quite right. I received the same results when I turn off the cache in CF8 and 9 it seams Railo is still king by a long shot with regards to object creation. ( in this limited test anyway )
@Rupesh Kumar
I have added the code that I used.
Just ran on OpenBD 1.1:
Objects: 100000
Seconds: 4.17
Objects per Second: 23980.8153477
This is OpenBD 1.1 on Tomcat 6.0.18 on a MacBook Pro running Ubuntu.
Just thought it would be funny if I could beat the any of the posted times on your blog Paul with Open BD / Tomcat / Apache / CrunchBang Linux on my Asus 1000HE netbook with an Intel Atom 1.6mhz mobile processor. So I ran your test code:
13.086 seconds for 100k objects
7641.75 objects per second
Looks that beats CF8 on mobile processing power. Anyways, it's a completely unfair comparison since I'm not matching CFML engine against engine, but it is funny to try to do it with an Atom processor.
I'm post tomorrow with my real rig using Tomcat / Open BD / CF8 / CF9 / Railo on Ubuntu.
@Peter
I was chatting to Matt Woodward about this and I was under the assumption that the servlet affects the speed of the CFML engine. Matt confirmed my suspicion and I feel a fair test you be to run all 4 Engines under the same servlet ( tomcat/jrun/glassfish etc ) this would be a really good comparison and I would really look forward to your results under Ubuntu.
The hinting results here might be leading to a performance decrease due to jrun and its not the first time I have heard of such gripe with jrun either.
@Paul, honestly I have no clue why they are keeping JRun around. It's not even sold any longer and there are far better open source servlets around these days.
Railo rules again ;-)
we have added the same feature (request based trusted cache for templates), cf9 has to railo 3.1.0.023.
with help of this feature railo beats cf9 again in every situation.
@Michael Offner
Well I just upgraded from .22 to .23 and in a virtual machine under glassfish, Railo now got as high as:
Railo
1.344
74404.7619
Impressive.
Here's a bunch more tests of a similar nature, all in a very controlled testing environment with a consistent servlet engine (Tomcat):
event:http://bit.ly/iaABJ
Oops, sorry, that URL had an "event:" prefix from TweetDeck :( Here it is:
http://bit.ly/iaABJ
You really should test with a Complex CFC with many functions (even simple set/get functions) as that has been CF's weakness in the past (since each function compiles down to an Object in java)
Post a Comment