Ideas

Too many variables involved that work against this idea
(mainly the fact that you are depending on individuals that have no business/employment interest with the projects. Thus the likelihood that the results might be skewed is very high)

however if you build an application that lets the end user download it and simply use it to run the check against

---> doing such you can compare multiple source feedback to assess how competent the tests were. One person could be wrong, but 100's of them testing the same product across the globe cant be wrong about the same thing at the same time.

Hey Gen

Thanks for the comment.

I agree with what you are saying.

One of the difficult parts of new mobile applications is that they often require network compatability. There are filters and specific ports from carriers which sometimes do not allow network access to servers on the web.

The full end functionality of the application is not really the test. (hopefully that has been tested already)

Does the application fit within the screen?

Can the enduser access the network part of the application?

If this is a location based application, do the maps show properly and is the end location correct or within predifined ranges.

Is the memory in the handset adequate?

Is the application quick enough on the specific carrier or is it painfully slow? Is it impossible to use at rush hour at the end location?

These are the types of things that drive application developers crazy.

I look forward to more comments or questions from you generic_idea_machine!

well imagine a "proxy scape"

An application that can you can "rent" out to the big guys (cariers) to use.

- So the carriers get to tweak and administer the back end of the application, specifying ports, mode of transfer, technology (gprs, edge, cdma, umts)

- and then you have the populace flock to the site and run tests as such. Based on the handset, the application redirects the user to the requisite test site.
- Based on the assessment, you have another application running in the back that would analyze the performance metrics. And provide the feedback back to the carrier.

- Carrier saves money by outsourcing testing.
- Beta testers make some dough in the spare time

Hey Gen

I hear what you are saying.

Application developers have emulators to test up to a certain point the application.

However in many cases it is the developer who has to support the end user application. There is no point in a finger pointing game to determine the cause for a problem on a end user problem.

It is the application developer ( or publisher) who has their reputation on the line. They are the one who need the application to work on a specific carrier with a specific handset.

A specific handset for one carrier may be completely different in OS than the same handset with another carrier.

The carriers do test up to a point but in many cases the applications are commodities to the carriers.

The statistical results of your "proxy scape" will give some parameters but in my personal experience, unless you test the specific application on a carrier handset, you cannot be sure it will work.

As well, carriers do not always give out test phones. It is up to the application developer to buy the phone. Try buying a phone in a country with a carrier without a home address in that country. Even as a developer it is virtually impossible.

I like the slogan "certified crowd tested", I think it would sell...

neat idea. Can even ask the "testers" to provide feedback on how to improve the application. Mind you, I see the "testers" as "users" much more than "testers". May be the crowd can even be pay extra bonus for catching bugs. And extra $ for meaningful features that the company is interested in developing.

Don't really like "Test 'er Test 'er good....". "certified crowd tested" is slightly better but there is probably a better one still.

Hey Kempton

thanks for the feedback....

Yes I believe application developers would love to get feedback in regards to GUI and other things.

It is way easier to make changes before a release goes out then when it is released.

There's definitley a lot of unique challenges when it comes to testing mobile apps and a solution that can address these will be a winner I think.

thanks drh

The challenges are very unique and expensive. The mobile application market is getting bigger and bigger.

According to analysts Juniper Research, the total global mobile entertainment market (including gambling, adult content, games, music and TV), is currently worth over $17bn.

The firm forecasts this will grow to $47bn by 2009 and $77bn by 2011, as broadcast mobile TV and mass market casual games really take off.

This is does not include business and other applications.

Taking it one step further .. how about a Mobile Developers Union? The union would be able to negotiate with both the carriers and equipment manufacturers on behalf of developers that are members of the union. Ok, crazy idea .. I know.

But back to this idea, I didn't really understand the proposed business model .. and I think I have a better one. The company harnesses a network of 100,000 mobile phone testers from all over the world, with every combination of carrier and phone. They offer several testing plans. For open source apps, they will run limited tests free of charge. For small businesses or individual developers, they offer cheap one-time or repeated testing plans, ranging in the hundreds of dollars. Or instead, this type of customer could join as a tester and get free testing in return. For big companies, the testing fees range in the tens of thousands of dollars.

The individual testers are not paid in cash, but instead they get "perks". The main perk will the ability to leverage the network for testing their own apps free of charge, but also to developer forums, online tools, early access to pre-released software, etc. When getting free testing
in return for doing testing, the system would ensure that members
do about 3x as much testing than they receive, in order to create
a profitable model.

Another arm of the business model would be advertising revenues to the developer and testers network.

Hey Branchcut

Branchcut Thanks for the feedback!

I like your business models and perhaps they would work.

I was looking at more of a bidding system where application developers and end testers would negotiate online to get testing done for a specific application.

For example, application developer abc says they will pay $50 to do a test on a specific model of phone on a specific carrier.

Tester xyz says they will do it for $75 dollars. abc and xyz negotiate until a price is reached. The money stays in escrow until the testing is done. On successful test, the money is released. The application developer pays a percentage fee to cell_tester (ie 15%) and the test is complete.

The more complex of a test, the more the tester will want.

The test procedures will have to be complete and upfront.

I have heard from carriers and application developers re this service and they both have agreed that there is a very definite need for this. Carrier testing labs cannot even get enough of the handsets to test the products.

In North America, there are very few open source mobile applications due to the nature of the garden wall mentality of the carriers.

I think a bidding system will ensure the right price for both the application developers and the testers. The market will even out for the appropriate application being tested and the complexity of the test required.

I think a paypal cash system will work out better than perks. The best testers will be rewarded because of the ranking system.

I am currently developing a mobile application and have had feedback from three testers. All three have different screen resolutions! I think this idea is excellent. Don't beta testers usually do it for free though?

Good idea, the benefits are obvious and potential for profit are clear.

My main concern like Generic mentioned early on would be guaranteeing the quality of service, due to human nature and if the tests will actually be carried out properly. The flip side of this is also if you’re an application developer who’s hired 20+ people globally are you going to bother rating your testers accurately and fairly?

I’m thinking that perhaps the best way to operate would be to provide the framework for testers to operate their own groups. Testers would set up groups of their known associates around the globe and bid for work as a unit and then be rated as a unit. A group of known associates would work better then many individuals as they will filter out the dross, as ultimately their rating requires everyone to work together.

Just a thought anyway.

I like the idea and think it has some good potential. In a magazine article Symbian claims that the USA is reaching the golden # of cell phone users, roughly 75%. In Japan and likely Europe, that 75% figure was a time when wireless carriers started needing application developers, not the other way around as it had been in the past. Those application developers are going to need this type of service. Down the road, if "Certified Crowd Tested" gains enough recognition the wireless carriers will look for the depth that this service could offer. BTW, the magazine puts the wireless data market at 16 billion - and this service could, in a sense, be a backbone for that industry.

I am currently developing a mobile application and have had feedback from three testers. All three have different screen resolutions! I think this idea is excellent. Don't beta testers usually do it for free though?

If the application developer can get the test done for free, (unlikely but possible) the service will still charge a minimum fee to the application developer. You have to remember that application developers spend thousands of dollars configuring various handsets for testing.

My main concern like Generic mentioned early on would be guaranteeing the quality of service, due to human nature and if the tests will actually be carried out properly.

I agree. Quality would be required. The groundwork for testing etc would be key.

The possibility using groups to test would also be possible..Ie a Sprint or Nextel group could test in those markets.

you folks have "thang" going on - back and forth, like that. it's great to see idea malignancy...in social theory, this is called particpant driven research - it is something that we participate in every day and it needs to become a more notable part of any business paradigm.

Hey wicker

Thanks for your comments...

Idea malignancy is an interesting phrase. What do you mean by this?

Certainly, from the latin malignant literally means "born to be bad" and is obvious for its cancerous meaning.

I look forward to your reply.

Much needed.

Example. I have on my P990i three WAP push messages - unsolicited advertising that when opened take you to a wap site. If I scroll them into view on my phone the message list crashes, with the error "not found"

I can't delete them, nor can I delete the 15 messages either side incase the rogue messages come on screen.

The handset manufacture, network operator and the message sender all have no idea why the messages are a problem or how to delete them - without deleting all my messages.

Better testing / standards adherance would benefit the whole mobile industry.

Hey Allan

Thanks for your comments.

I totally agree.

This idea would likely appeal to the youth market. Perfect for downloading and testing free games. Give them incentive to be a "tester" by offering free minutes (if appropriate) or points to get free stuff which would be provided by whoever wants their mobile product tested. Kind of like a "Nielsen Household", it could become a "privilege" to be a tester.

Thanks for the comments smoir1.

You are absolutely correct that this could be used as a marketing tool as well for the application developer.

Market and functionality tests would be great for feedback.

I attended a lecture from Steven Cheng of Innaworks who deals with this problem and many other issues surrounding this market. Steven ports tests and optimizes for essentially every major software companies in the mobile arena. Some of the insight I gained:

Its a hard slog and the carriers are god, they control everything with a horrific manner.
This is a highly competitive market. Often they will make versions of a cellphone app for every popular model.
Sending it to a porting house in India is cheep.
Relying on users to provide the right feedback would be tough.
Unlike computer software betas, buggy cellphone software is not fun!

Hey Patmania

Thank you for your comments.

I will try to answer your concerns.

"Its a hard slog and the carriers are god, they control everything with a horrific manner."

I know and sometimes the carriers do not test adequately. The testing labs sometimes do not have all the handsets to test. (I have had emails from mobile operators who would love this service because the handset costs come out of their departmental budget)

"This is a highly competitive market. Often they will make versions of a cellphone app for every popular model." True but we are not talking about the cost of porting (ie programming for various models) but the cost of testing can be huge especially with networked and location applications.

"Sending it to a porting house in India is cheep" True and that is why testing costs so much. You cannot test a USA Sprint handset application in India because Sprint does not operate in India. (ie the handset application may require specific networking parameters and handset parameters which are common to the USA operations only) A networked application cannot be tested in India because the network will be different.

"Unlike computer software betas, buggy cellphone software is not fun!" That is why the testers should be paid by the auction system. I anticipate testers getting paid to do their job. If a particular test was really tough, the tester will bid higher to do the work, the next time. Let the market decide what the costs will be.

I hope this answers your concerns.

Take a look at the business at http://www.cambrianh...ss/view/cell_tester/

quick questions:
-will you support Java as well as BREW?
- since you mentioned it's an auction, I assume testers will bid to do
a testing job, and cell companies will accept the lowest bids? A model
which increases revenue for testers would be beneficial to them
- what constitutes testing? what are the methods, metrics etc
- how will vet testers' experience, abilities and qualifications?
- how will you verify that the work was done, before payment?

Nifty idea fish! Good luck with it!

Answers to TheGuru...thanks for those.

quick questions:
-will you support Java as well as BREW?

Whatever the endusers phone will support will be tested. If it is brew or j2me or symbian

- since you mentioned it's an auction, I assume testers will bid to do
a testing job, and cell companies will accept the lowest bids? A model
which increases revenue for testers would be beneficial to them

the bidding will go both ways. Let the market decide the price to do the job.

- what constitutes testing? what are the methods, metrics etc

The methods etc. will be the application developers responsibility. The testers will see the methods etc and bid what they believe to be the appropriate cost to do so.

- how will vet testers' experience, abilities and qualifications?

The testers will be ranked by the application developers and vice versa. The testers will show what experience they have online.

- how will you verify that the work was done, before payment?

The application developers will determine the work to be done. If it is a network application they will see the traffic. The money will be kept in escrow until both side are happy with the work.

I hope this answers your questions.

Great Idea!

Keep going!

They all ready do this thing called beta product testing. So, there is no way any one would buy this "service" that they are all ready are doing!

The problem with beta testing or any testing is that the application developers cannot get the phones to test without paying huge amounts of cash. (if they can get the phones at all)

If they are buying the phones with the plan is is because they need a technical to test not the average home user. Usually they get review models from the companies at no cost.

For beta testing usually they pay groups of people next to none to get people to try a program for free. Then they will usually send them a free copy of the program after they release the full copy.

So, I don't see why they would want to use this service when they all ready do it themselves.

like it!

Nope

I'm in development of mobile games. I can tell you now it doesn't work at all like you think... Companies want a specific game on specific phones and that's it.

The games give user a reason to buy a specific phone and at the same time, if you want to play the new version of a game or app.... You have to buy a new phone.

Your idea will never work until companies are ready to change their ways... with I don't.

Sorry about this.

]V[oogy

robertman & Kevin_Cox: I think you are missing the point here.

Sure, if you're Microsoft you can get beta testers easily but for those of us with limited resources this service will be a godsend.

The blastfm alpha is currently only confirmed as working on two different devices, both of them S60 3rd edition. There are almost as many different j2me implementations as there are devices and they all differ in subtle ways particularly regarding the mmapi.

I will be happy to pay for this service when it launches and there are many other small developers who I believe will pay also.

Hey Moogy

Thanks for your comments.

I have to disagree with you. I am the former president of Blister Entertainment (first mobile location based games in North America)

The difficulty of testing games on phones is difficult because it is hard to get the phones on a specific network.

I have had responses from the testing departments at mobile companies who applaud this because they cannot get the phones themselves.

This is especially true on networked games and applications.

Remember this is a cost and leverage business. The average cost of testing on a specific phone

Remember this idea will reduce the cost of testing.

"The games give user a reason to buy a specific phone and at the same time, if you want to play the new version of a game or app.... You have to buy a new phone."

Yes this is true. However if you wish to test on a global basis, you need to test in each market on different phones with different networks.

Very expensive.

Why not use the handsets that people already have and have them test. You do not have to fly teams of people out to test in a market. Remember,

Much cheaper.

The slogan says it all..."CERTIFIED CROWD TESTED"

BTW I have downloaded multiple mobile games on my handset, that were buggy and in some cases did not work at all.

I don't have a good knowledge about mobile applications but this idea seems to be a great tool to resolve some incompatible issues… It has my vote!!

fish, I'll give you a five (since I usually only give a 1 or 2) since your idea is:
1) actually viable 2) there is a need 3) people are willing to pay. to top it off you answered all of everybody's questions intelligently, showing you'd thought this through. This is by far the best idea I've seen, and is leaps above most of the crud that is on here. I hope this idea makes it on it's own merit, and competes with the technically flawed and BS ideas that have floated to the top. Don't forget - the cream is not the only part that rises to the top! :(

Hey TheGuru

Thank you for your kind comments.

I do disagree with you on 1 point however. I do believe there are good ideas out there that are possible and feasible. The difficulty with some of the ideas is the communication of what they wish to do.

I wish all the ideas a good chance and wish them all good luck.

Great idea and unlike most submitted on CH, could actually be done without a multi-million dollar budget and a few spare years of product development time.

If you need a hand in Australia, let me know. I am guilty of always buying the latest and greatest nerdy smart phones thanks to my passion for LBS.

me gusta

Sounds like a good idea. I can't believe there is a gap like you describe, it's almost bizarre. So many users would be willing testers, it's logical to use them for it.

What I don't understand is why anyone would (previously) have flown testers to various locations to do the testing on local technology.

Why can't a product developer simply place an ad on whatever medium will reach the intended market, and get testers to sign up remotely?

If all it takes is a phone with a local account, why was it necessary to fly people anywhere?

There is something missing in my understanding of the scenario, but as I understand it the solution is simple and sensible.

I guess if you are creating this network of testers as a stand-alone business, then you are not linking it to just one particular product. The tester network could be used for all kinds of market and beta testing, if it became large enough.

I have never heard of any companies flying people out to a location just to beta test some software. My guess is that there were other reasons "mobile location based games".

Software companies that I know for mobile simply use the review units by the phone distributor or have what is know as a beta test. Basically like I said before. Usually they pay groups of people next to none to get people to try a program for free. Then they will usually send them a free copy of the program after they release the full copy.

A lot of the of the time they don't even pay them, just offer them something as simple as some pizza and drinks.

I think most of us are familiar with the concept of beta testing. If you are huge already or are willing to do a huge marketing pre-launch effort a la Joost then beta testers are easy to find. But, with an new product it is not so easy to get it out there.

Hi Jill

Thank you for your comments.

"Sounds like a good idea. I can't believe there is a gap like you describe, it's almost bizarre. So many users would be willing testers, it's logical to use them for it."

The mobile space is a strange place. It is not easy to find testers in various markets. Remember this is a global market with various carriers. Each carrier is different. Every phone is different.

"What I don't understand is why anyone would (previously) have flown testers to various locations to do the testing on local technology."

Imagine you are an networked application developer in Toronto. If you wish to test an application for Sprint in the USA, you cannot test the application in Toronto because your phone would not be on the Sprint network. You would be testing likely through Bell Mobility or Telus in Toronto. Each network is different.

"Why can't a product developer simply place an ad on whatever medium will reach the intended market, and get testers to sign up remotely?"

Well the difficulty would be, where would you place an ad? What are the timing issues with this? This idea would essentially do this.

"If all it takes is a phone with a local account, why was it necessary to fly people anywhere?"

Here again if you are an small application developer in Toronto, try to get a phone on the Sprint network in the USA without a US address. If you cannot show an id with an American address, Sprint will not sell you a phone with a plan. As well, if you did have an address, the phone may cost up to $1000 with a plan. Then you need people to go the US to test the application. The test may only be an hour long, but it has cost thousands of dollars to test the application.

"I guess if you are creating this network of testers as a stand-alone business