When & Where Do You Use Mongo And Nodejs

[semaphore]

Basically mongodb has greater performance than MySQL in large tables, we have had to start caching some mysql tables because queries can take minutes with millions of rows. Nodejs goes to mongo as php goes to mysql, but you can cross use. Mostly preference and app database size

Queries that take minutes with a million rows are either inefficient, badly designed, missing crucial indexes or running on bad hardware. Our transaction table is a few million and it doesnt take more than few seconds to query.

 

[koeksGHT]

Queries that take minutes with a million rows are either inefficient, badly designed, missing crucial indexes or running on bad hardware. Our transaction table is a few million and it doesnt take more than few seconds to query.

The tables are quick, the reporting behind loading multiple queries together gets a bit complex querying multiple servers. Therefore a cache and cron running keeps things quick and is easier for graphing and tools.

 

[shauntir]

After reading even more about Nodejs, it becomes really difficult to know when exactly to use it.

Hard-realtime software, no no, go Erlang. Then, others say, if you use relational dbs, use another more common tech stack (rails, mvc, django, php). Some say, even templating could block the thread if it serves vast amounts of markup - that then makes something like express moot...

Oh how I love new technology.

 

[semaphore]

After reading even more about Nodejs, it becomes really difficult to know when exactly to use it.

Hard-realtime software, no no, go Erlang. Then, others say, if you use relational dbs, use another more common tech stack (rails, mvc, django, php). Some say, even templating could block the thread if it serves vast amounts of markup - that then makes something like express moot...

Oh how I love new technology.

Well those people are idiots. I have not ever had a template block my requests. The people that get blocks like that clearly dont understand how to use the frameworks or even understand how nodejs works.

app.get('/', function(req, res){
  res.render('index', {
    title: 'Home'
  });
});

app.get('/about', function(req, res){
  res.render('about', {
    title: 'About'
  });
});

app.get('/contact', function(req, res){
  res.render('contact', {
    title: 'Contact'
  });
});

There template rendered.

 

[shauntir]

Well those people are idiots. I have not ever had a template block my requests. The people that get blocks like that clearly dont understand how to use the frameworks or even understand how nodejs works.

app.get('/', function(req, res){
  res.render('index', {
    title: 'Home'
  });
});

app.get('/about', function(req, res){
  res.render('about', {
    title: 'About'
  });
});

app.get('/contact', function(req, res){
  res.render('contact', {
    title: 'Contact'
  });
});

There template rendered.

Nicely put. I've also been trying to figure out how it would block since if you're rendering a page, there shouldn't be any logic aside for simple foreach loops and/or if/else. That's about it...

 

[CheekyC]

yep i am using both.
no my relational model doesn't keep data in couch.
i use standard sql via entity framework for my data.
so the site runs off of sql as standard.

but for reporting, and keeping track of whats happening on my site i use a couchdb instance.

i have a document "object" i call an activity.
an activity is a json structure that has a datetime, a name, a description, a type, a few other things and then it also has a payload. my payload is another json object, and its shape depends on whatever activity it is you're doing.

so when you perform a review, i might save the actual review object as my payload.
or when you perform a search, i might save the search queries (the words, the dropdown values etc) as the payload.
or when you view a particular tag, i might save the userID and the tagID as the payload.
the point is just this allows me to save anything really, along with the date and time it happened.

if i did this in sql, i would have to store these as blobs or something. which sucks ass coz you can't query it.

using couch i can write a query on the activity itself, so maybe view all activities on this date, or on this hour, or whatever.
or i can query all activities where the datetime is this, and the type is "search" and the payload.searchwords is something else.
i can query the payload directly.

also, querying your reporting db might have a negative performance impact, whereas couch i'm making async http posts or gets so sql isn't affected at all and all the other users on the site won't feel it.

you "use" couch through json.
because my couchdb is a hosted instance, i interact with it via async http calls.
its not the same as the way i interact with entityframework.

entity framework uses c# classes, whereas i do have one class for my activity table, but i tend to use a lot of dynamic objects when querying the payloads.

DING! Aha moment. I could never really get my head around the ned for doc centric dbs. I have always used relational dbs. But querying non structured data makes lot of sense now. Using doc centric for your code instrumentation is a good idea

 

[semaphore]

DING! Aha moment. I could never really get my head around the ned for doc centric dbs. I have always used relational dbs. But querying non structured data makes lot of sense now. Using doc centric for your code instrumentation is a good idea

Cassandra is good for that
http://www.datastax.com/dev/blog/advanced-time-series-with-cassandra

 

[noob_saibot]

The best example you can get for when to use NodeJS is something like a web-chat app.

This next statement may rile some people, but lets not burn any bridges here. I would say NodeJS is useful for tasks that aren't too intense on the CPU but would suit the non-blocking IO and multi-threaded situations (eg. the chat app).

There's actually a lot that goes on in Node land everyday and if none of us are on the bleeding edge, then anything we say that was true yesterday may no longer be true today.

As far as using Node and Mongo goes, I would consider using something on top of them (like Express or even Meteor - I personally like Meteors approach, which seems to combine all those lovely parts to make a decent and quickly-built app with a backend too).

What I find strange is that so few okes here are exposed to that kind of technology (like deeply involved with Node, although I definitely think many guys are using Mongo or another non-rel DB).

Release the Kraken!!! ( I just felt I had to say that)

 

[koeksGHT]

The best example you can get for when to use NodeJS is something like a web-chat app.

This next statement may rile some people, but lets not burn any bridges here. I would say NodeJS is useful for tasks that aren't too intense on the CPU but would suit the non-blocking IO and multi-threaded situations (eg. the chat app).

There's actually a lot that goes on in Node land everyday and if none of us are on the bleeding edge, then anything we say that was true yesterday may no longer be true today.

As far as using Node and Mongo goes, I would consider using something on top of them (like Express or even Meteor - I personally like Meteors approach, which seems to combine all those lovely parts to make a decent and quickly-built app with a backend too).

What I find strange is that so few okes here are exposed to that kind of technology (like deeply involved with Node, although I definitely think many guys are using Mongo or another non-rel DB).

Release the Kraken!!! ( I just felt I had to say that)

Haha yep, we are using Nodejs for a real time chat app. Very slick to use.

 

[noob_saibot]

Also, if you need to see an ultimate fail video, check this guy out:

www.youtube.com/watch?v=1e1zzna-dNw‎

I particularly like the "sloooowwwww" (as if he is doing a slow-play in realtime )

 

[semaphore]

The best example you can get for when to use NodeJS is something like a web-chat app.

That is the most common example, but i would not say its only NodeJs's strength. You can achieve that very easily with RoR, PHP, Play and ASP.NET as well.

This next statement may rile some people, but lets not burn any bridges here. I would say NodeJS is useful for tasks that aren't too intense on the CPU but would suit the non-blocking IO and multi-threaded situations (eg. the chat app).

To a point you're right but not entirely, you can write cpu bound tasks and still get away with non-blocking. You just need to be clever about it, but yes i would not write encoders in nodejs.

There's actually a lot that goes on in Node land everyday and if none of us are on the bleeding edge, then anything we say that was true yesterday may no longer be true today.

Not really. (i.e. express) obviously there are changes in methodologies but what is today wont be tomorrow is not entirely accurate.

As far as using Node and Mongo goes, I would consider using something on top of them (like Express or even Meteor - I personally like Meteors approach, which seems to combine all those lovely parts to make a decent and quickly-built app with a backend too).

Meteor is a load of ****. And the code base is so immature.

What I find strange is that so few okes here are exposed to that kind of technology (like deeply involved with Node, although I definitely think many guys are using Mongo or another non-rel DB).

Its because new technology adoption rates are very slow in SA, especially when it comes to JS. People are still under the impression JS is still only on the web client domain.

 

[koeksGHT]

Still not a huge fan of node. Overcomplicated IMO for stuff that can be done easily and quickly in other languages such as PHP/javascript etc. definitely has its benefits tho

 

[noob_saibot]

Meteor is a load of ****. And the code base is so immature.

Can you explain why Meteor is bad?

It looks very useful, but I haven't played around with it extensively yet.

 

[Waansin]

We (Mxit's Web Team) are using MongoDB in a production website that served about 11 million page views for February. MongoDB offers a lot of benefits beyond a typical RDBMS, but the immediate downsides include needing a lot of servers to get replication and sharding to work effectively and needing more storage.

From a developer's perspective, MongoDB is almost child's play. Fun to use, but almost too easy to accidentally break though. We do store unstructured data mostly and using only a single data store makes life a bit simpler.

Certain types of queries can be difficult to optimise. We have had to perform large migrations of data to optimise sharding as the data grew.

The learning curve to understand how best to use a document store database like MongoDB can be a bit longer than expected. This is especially true if you have only ever thought about data storage in terms of structured data sets.

MongoDB is working well for this website and its set of services, but your mileage may vary. Internally, we have used a variety of languages (PHP, Python, Nodejs*) and a variety of data stores (MySQL, Postgres, MongoDB, CouchDB) in order to meet our differing requirements.

* Fully aware this is a platform. Please substitute any words and phrasing to make it proper for you

 

[semaphore]

Can you explain why Meteor is bad?

It looks very useful, but I haven't played around with it extensively yet.

Meteor is a "play" framework, no serious company will develop a product on it because it offers out the box support for web sockets, that also means its oppionated and they force you into what they think is the correct way of using it (which i have a problem with). Its fun to play with, fun to learn from and that is where its use in my eyes end. The best way to learn and understand system architecture is to build from the ground up and not use pre-existing toolsets that do everything for you, because when there are problems and you dont understand the underlying architecture/framework you will suffer to support it.

Also the fact thats its on a preview 0.x.x release is a bit thing as well.

 

[semaphore]

MongoDB is working well for this website and its set of services, but your mileage may vary. Internally, we have used a variety of languages (PHP, Python, Nodejs) and a variety of data stores (MySQL, Postgres, MongoDB, CouchDB) in order to meet our differing requirements.

Sorry i know im nit picking here, but nodejs is not a language its a platform.

 

[Waansin]

I tried to edit my response to indicate "Javascript on the server" or "Server-side JS using Nodejs" and even tried "Javascript with Nodejs." The thing is, none are as neat and tidy as thinking people will make the leap beyond nitpickery and accept that the use of a platform's name can indicate the language involved within that platform. Because, frankly, while Nodejs uses Javascript as a language, it is possible that this Javascript is conceptually quite different to what people typically believe Javascript is. Not that this assumption is even true.

In other words, we're all predisposed to pedantic extremes. But, mainly because it seems to upset you more than me, I am not going to edit my post to make it pedantically correct. Oh, alright. I added a footnote.

 

[semaphore]

I tried to edit my response to indicate "Javascript on the server" or "Server-side JS using Nodejs" and even tried "Javascript with Nodejs." The thing is, none are as neat and tidy as thinking people will make the leap beyond nitpickery and accept that the use of a platform's name can indicate the language involved within that platform. Because, frankly, while Nodejs uses Javascript as a language, it is possible that this Javascript is conceptually quite different to what people typically believe Javascript is. Not that this assumption is even true.

In other words, we're all predisposed to pedantic extremes. But, mainly because it seems to upset you more than me, I am not going to edit my post to make it pedantically correct.

Thanks in essence you just did.

 

[_kabal_]

Nodejs also is a "preview" 0.x.x version

 

[_kabal_]

Still not a huge fan of node. Overcomplicated IMO for stuff that can be done easily and quickly in other languages such as PHP/javascript etc. definitely has its benefits tho

I am also not a fan of painting with a brick. Wait, that is a silly statement...