Adapting and Using Existing Data with Sencha Touch 2

This post is a follow up to my talk at London Sencha Touch User Group on 11th July. The talk was about the steps I took to adapt the existing events data on to be used by the Sencha Touch 2 app I have developed to list Artsfest events in a more mobile friendly way. The prototype for this is now at I will go through the following in this post:

The Problem

1. How to adapt and reduce existing data to the data you actually need

2. How to import this data into a MongoDB database

3. How to access the data through RESTful urls in a Node.js app

4. How to set up a MongoHQ database and export the data to it

5. How to put the Node.js app onto heroku p

6. Example of a Sencha Touch2 app calling these urls as the proxy for the data.

All code is available on my github. For the data setup up on MongoDB please look at the sourceJSON. For the Node.js app its the Nodejs.Code repo and finally for the Sencha Touch code, as you might have guessed, its the SenchaCode repo.

The Problem

There are lots of examples of how to build demo apps from the ground up for Sencha Touch 2 (ST2). But what if you are using data from an existing website or application and you cannot pull the data direct from the existing site (as is the case with Artsfest). So this post presumes you have no control over the data. You are simply acquiring it and using it for your own means. This was the dilemma I had when creating my ST2 app. There are several issues at play here, the most important are:

– Options for data to be served to the ST2 app:

The choices you have (as far as I’m aware) for data formats that can be used by Sencha are YQL, json or using AJAX.

– How to convert the source data to what you want to serve to the ST2 app.

For this project the choice of data is fairly straightforward as the artsfest website can feed json data, so I went with json to supply to the ST 2 app as it wouldn’t then take much to convert the existing data to that which is more suitable for my needs. But what if your data source is a SQLServer or MySQL database? To be honest I haven’t looked too much into how to convert conventional database data to JSON. It seems with newer technologies such as this MVC example you can convert to JSON quite easily. But this presumes you have control over the complete back end, a luxury I don’t have. A quick google found this for a SQL Server solution and this for a MySQL solution of converting to json. More thorough research, I’m sure, would reveal possibly better options. Anyone with suggestions for this or something to add, then please leave a comment.

1. How to adapt and reduce exisiting data to the data you need kindly provies urls to obtain pure data from the site. The main url is here, which details several ways to obtain the data in dfferent formats.

As mentioned, for this project I wanted data in a json format so used a url similar to the example (
The data returned was everything I required but there was too much data. A lot of repetition. I only wanted a fraction of what was supplied for a more lightweight database for the mobile web app.

For the app I wanted two Data Entities

Markers – the locations where events are hosted
Events – each one would have a marker ID

To reduce the data I used a .NET JSON parser, which I adapted, compiled and executed locally. This gave me the two JSON files, one for each entity. The parser basically picked up certain fields I thought were most relevant (i.e. Events had fields like description, title, date of event, Markers had fields like long, lat, title, description) and put these in the new json, and ignored the rest. I won’t go into more detail on this process as this is not the focus of the demo. If you are interested in this process then please get in touch.

2. How to import this data into a MongoDB database

Now I have a reduced dataset I want to import the data into my local MongoDB database. First of all you must ensure that each row (document in json world) should begin on a new line in each json document or this process doesn’t appear to work. Once you have the files formatted, you are ready to import them into MonbgoDB. I’m presuming at this time you have mongodb installed locally. To do the import, open a terminal window navigate to the folder containing the events.json and markers.json files, then execute the two commands:

$mongoimport -db demo -c markers markers.json
$mongoimport -db demo -c events events.json

…where demo is the mongodb database and -c argument is the collection and then finally the json file you are importing from. You should get a message in the terminal similar to:

connected to:
imported 137 objects

This confirms that the data was imported. Note that I didn’t have to create the demo database beforehand. One of the great things about MongoDB is the way you can implicitly create databases and collections without any setup. To test the import went ok and to view the database, open a separate command window and execute “mongo” to open a connection to your local instance of mongoDB. Once you are connected, switch to your newly created “demo” database by using the mongo command “use demo”. To list your markers, execute “db.markers.find()”, to list events; “”. This query reference gives you more query ideas.

3. How to access the data through RESTful urls in a Node.js app

So the MongoDB is working ok. The next step is to access the data through a Node.js application. To create a Node.js app is not as daunting for a complete newbie as it sounds thanks mainly to the express framework. I recommend installing it then following the terminal commands below to instantly create a Node.js app that you can use straightaway.

alex@ubuntu:~$ express /tmp/foo && cd /tmp/foo

create : /tmp/foo
create : /tmp/foo/package.json
create : /tmp/foo/app.js
create : /tmp/foo/public
create : /tmp/foo/routes
create : /tmp/foo/routes/index.js
create : /tmp/foo/views
create : /tmp/foo/views/layout.jade
create : /tmp/foo/views/index.jade
create : /tmp/foo/public/javascripts
create : /tmp/foo/public/images
create : /tmp/foo/public/stylesheets
create : /tmp/foo/public/stylesheets/style.css

dont forget to install dependencies:
$ cd /tmp/foo && npm install

alex@ubuntu:/tmp/foo$ npm install
jade@0.26.3 ./node_modules/jade
├── commander@0.6.1
└── mkdirp@0.3.0
express@2.5.5 ./node_modules/express
├── mkdirp@0.0.7
├── mime@1.2.6
├── qs@0.5.0
└── connect@1.9.1
alex@ubuntu:/tmp/foo$ node app.js
Express server listening on port 3000 in development mode

This can then be adapted by editing the app.js file so that you have RESTful routes (urls) to access the MongoDB data with. The app.js file can be found in the Node folder on the github repository. Once you start the node.js app locally, (and ensuring you have the mongoDB databse demo running) the routes provided to access the data are :

http://localhost:5000/markers?callback=callback1 to return all the markers
http://localhost:5000/events?callback=callback1 to return all the events
http://localhost:5000/events/7?callback=callback1 to return events for a specific markerid (7)

4. How to set up a MongoHQ database and export the data to it

Now you have the data working for you locally, you can set it up to be accessed over the web. MongoHQ provides hosting for MongoDB databases and best of all there is a free “sandbox” option for the hosting of each database.All you need to do is sign up, create a database (I created a database called “senchaDemo”) and then set up a new user (“test”) with a password. Next, go to the Database Info tab. Just make a note of the Mongo URI such as mongodb://, which can then be used in the mongoose.connect call in your Node.js app.js file. Next thing to so is to export the data from your local MongoDB database to this new database. The instructions are on the MongoHQ site, so I won’t repeat them here. The final thing to do is to hook the local node.js app up to your new remote MongoDB by changing the mongoose.connect connection string from local:


to a remote connection


where test is the mongoHQ database user I set up and test is its imaginiative password.

5. How to put the Node.js app onto heroku

Now the data is online and ready to use, the next step is to put your node.js app onto heroku or any cloud based service but I am going with heroku as its very reliable and free to use the basic service. The first thing to do is to add the node.js code to a git repository. I always forget this process. This blog post is as good as any for showing you what to do on your local machine, beginning with git init. Once you have created the git repository you can push it onto heroku. First create the remote application with this terminal command (responses from heroku also included:

alex@ubuntu:~/NodeSenchaDemo/Node$ heroku apps:create -s cedar
Creating high-frost-8468... done, stack is cedar |
Git remote heroku added

next push to heroku

alex@ubuntu:~/NodeSenchaDemo/Node$ git push heroku master

then you must scale the process

alex@ubuntu:~/NodeSenchaDemo/Node$ heroku ps:scale web=1

then you can check if the process is running correctly

alex@ubuntu:~/NodeSenchaDemo/Node$ heroku ps

which will give you the output of

=== web: `node web.js`
web.1: up for 10s

you can also check the log files to see if they report any issues
alex@ubuntu:~/NodeSenchaDemo/Node$ heroku logs

…hopefully they won’t and now you have a working node.js app hosted in the cloud.

6. Example of a Sencha Touch 2 app calling these urls as the proxy for the data.

Now we have the data supplied through the Node.js app, we can use it in a ST2 app. Sencha Touch 2 can use JSON or JSONP as its proxy. I have put the ST2 demo application that uses the data at and the code can be found at github. The Markers.js store file shows how to set up the proxy file…

Ext.define('', {
extend: "",
config: {
storeId: 'markerStore',
model: "demo.model.Marker",
autoLoad: true,
proxy: {
type: 'jsonp',
callbackKey: 'callback',
url: '',
reader: {
type: 'json',
rootProperty: 'data',
successProperty: 'success'

Note the type is ‘jsonp’. JSONP, enables you to get data from a different domain/server. So we can get the data from the heroku app while hosting the ST2 app on a different domain.

The demo ST2 app is a simple tabPanel application to show how to call each one of the three urls that the Node.js app can supply. If you are completely new to Sencha Touch, this video will give you the basics on tabPanels.

I hope you find this info useful. If you are scratching your head over anything in this post or need more info then please leave a comment. Any feedback would be appreciated.


3 thoughts on “Adapting and Using Existing Data with Sencha Touch 2

    • Hi Shane. Glad to hear it helps you. Mongoose is a driver to enable node.js to connect to a mongodb database. As you can see in the node.js app.js file, it connects you to the database and creates references to the models which you then use to get the data. If you google node.js and mongoose I’m sure you will see a few tutorials that go more in depth. Hth, cheers . Alex

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s