Multi-client enviroment, 1 database

G

guest2013-1

guest
Joined
Aug 22, 2003
Messages
19,800
Reaction score
13
How would you approach a small cluster of users working on a program, that when 1 updates a record it immediately reflects on the others, almost notifying them that this record has changed by user xyz?

I'd like to avoid having to use the sync framework or write my own conflict resolution scripts.

Is there a way for the sql db to tell a client to refresh it's data? Or is polling my only other option?

I was thinking of maybe having the client's broadcast to each other that record xyz changed and then update with the changes without hitting the db. Or even giving specific records to specific users responsible for editing them and not letting anyone else edit those records except maybe an admin?

Any idears? :p
 
Having clients broadcast would work - but you don't want clients to be aware of each other. The problem you're describing is usually solved by PubSub. All your clients subscribe to a channel - and any time your service handles a a change, it'll publish to the channel which will notify all clients. How you handle the changes is up to you - delta changes are always best, but from my experience become more difficult when you deal with more complex data structures.

AcidRaZor said:
How would you approach a small cluster of users working on a program, that when 1 updates a record it immediately reflects on the others, almost notifying them that this record has changed by user xyz?

I'd like to avoid having to use the sync framework or write my own conflict resolution scripts.

Is there a way for the sql db to tell a client to refresh it's data? Or is polling my only other option?

I was thinking of maybe having the client's broadcast to each other that record xyz changed and then update with the changes without hitting the db. Or even giving specific records to specific users responsible for editing them and not letting anyone else edit those records except maybe an admin?

Any idears? :p
Click to expand...
 
is this a web/browser app, or a "desktop" app?

Web app could use HTML 5 WebSockets, although this would limit what browers were fully compatible with your app

If "desktop" apps, publish-subscribe messaging should work just fine.
 
Last edited:
I used ajax and a counter to check every couple of minutes if there is a data change. if yes, fire off an event to notify the user he must refresh his page. You cant refresh the page automatically as he would loose whatever he is doing at that exact moment the page refreshes.
 
From a design point of view - it's not the client's resposibility. The client simply consumes and edits data. It shouldn't be responsible for managing the state of other clients.

If there's any scope for the number of clients to increase, you need to consider the stress this places on the client. Your service layers are always scalable - the client is not. A client will struggle to hold n number of sockets open and broadcast to everyone of them as n increases.

Also, Websockets isn't a solution any more than sockets are a solution; you still need some sort of message orientated paradigm like PubSub to handle communication over the sockets. If it is a browser issue, SockJS or socket.io will give you support for most browsers.


AcidRaZor said:
Why wouldn't I want clients to be aware of each other?
Click to expand...
 
I'm using rabbitmq for my message brokering. Have multiple consumers and publishers that all listen to specific types of messages

its a basic push notification mechanism you want. Which is pubsub in essence

I can make a demo app for you if you want, if its web based you can use signalr
 
Last edited:
terencej said:
I used ajax and a counter to check every couple of minutes if there is a data change. if yes, fire off an event to notify the user he must refresh his page. You cant refresh the page automatically as he would loose whatever he is doing at that exact moment the page refreshes.
Click to expand...

use a comet approach instead.
 
AcidRaZor said:
Why wouldn't I want clients to be aware of each other?
Click to expand...

sorry for the quotes driving and on mobile, firstly why would you ever want a client aware of another instance. That has bad design written all over it
 
I'm just playing around with ideas. Thanks for the suggestions. I will look into it. It's a Desktop App for now. There's been some talk of pushing it into a web based app but I feel it's not going to really fit well into their current environment (even though they have a big web app for their main systems)

With this I'm also busy learning WPF as I go a long. Their current front-end is MS Access connecting to a SQL Server database, and it's rife with data errors. Felt like it will be a nice side project to my current duties, especially during the slower month in December (not taking leave)

The push notification idea (pubsub) I haven't thought of, but might just work. It's not a high volume environment. Currently have about 12 users and should stay that way. The amount of data they process will become more and more in the future, that's why I want better controls of data they put in (since it affects the reports a great deal). Plus giving them a better UI, they can quickly spot open tickets that haven't been closed yet by night shift etc
 
hof said:
From a design point of view - it's not the client's resposibility. The client simply consumes and edits data. It shouldn't be responsible for managing the state of other clients.

If there's any scope for the number of clients to increase, you need to consider the stress this places on the client. Your service layers are always scalable - the client is not. A client will struggle to hold n number of sockets open and broadcast to everyone of them as n increases.
Click to expand...

Of course it is not the clients responsibility to notify other clients or even be aware of them, and neither are nessesary. The client updates the server, and the server publishes to that topic that data X has changed or data Y was added (handing merging and conflict resolution once you know your working data has changed is a whole other story :) )

This allows for virtually unlimited clients.

This is how we successfully implemented a real time car hire system in Delphi years back

hof said:
Also, Websockets isn't a solution any more than sockets are a solution; you still need some sort of message orientated paradigm like PubSub to handle communication over the sockets. If it is a browser issue, SockJS or socket.io will give you support for most browsers.
Click to expand...

Sorry, I assumed it was a given that there would still be pub/sub mechanism driving this process
 
Last edited:
The only reason why I thought of a torrent-like update from client to client that data changed on the server was that not all operators will work on the same data, but if they did, they'll get an instant notification without anything having to run on the server.

as far as I understand, pubsub/push notifications still require some server-side thing to run and check these things, unless you can tell me that sql server itself can push a data-update notification somehow (which I doubt)

Since this will run on a local network, connecting to a local server and the network being up all the time, I think this might actually not be a bad idea of, when client A updates record 3, and record 3 being open on client B, that client B would be notified of the change that was committed to the DB before they could save their record. (The save button would disable for example)

Each operator is usually responsible for their own set of tickets that are open, so this shouldn't happen at all, but I'm catering for all types of scenarios that MIGHT happen so that there would be no need for conflict resolution and user A can't override user B's work (which sometimes happen) and then the manager breathes down everyone's neck trying to figure out who the dumbass was :)
 
You could always just add a lock field to the record which only enables read access to the data.
 
Yup I could, but then you have to start managing lock states and I'm pretty sure we'll have quite a few support calls to "unlock" records. And since they're 24/7 and I'm not, I'd like to keep support calls to a minimum. For some reason the web app I mentioned also does a lock like this so only 1 user can work on the data and it's a nightmare doing support, because either I can't enter the system because someone is locking it, or just by logging in and closing my browser, I locked everyone else out from doing their job.

These users are sneaky buggers. The other day we found out one of the sales people found a loophole in the way they quote clients (they're not allowed to re-use rates quoted to another client as each client is different and there's some legal mumbo jumbo mentioned as well). Instead of letting IT know there's a way to dupe the system, they've been at it for *years*. It was only recently when we rewrote a system to add new processes to the flow of data that we encountered this nice little bug.

It's been a few weeks now that there hasn't been a support call for "why can't I submit this for invoicing" questions, so they are nice and locked down now :)

The other day we found they found another loophole in the ERP system where they're able to change dates on items that shouldn't have their dates changed at all. So billing is able to invoice before a job has been done and when the job arrives on site, they can't find it because billing already "completed" the job for them. *******s :D
 
AcidRaZor said:
The only reason why I thought of a torrent-like update from client to client that data changed on the server was that not all operators will work on the same data, but if they did, they'll get an instant notification without anything having to run on the server.

as far as I understand, pubsub/push notifications still require some server-side thing to run and check these things, unless you can tell me that sql server itself can push a data-update notification somehow (which I doubt)
...
Click to expand...

You would create a topic for each silo of data:

pubSub.subscribe("/document/1653623", callback);

A client will only recieve notifications about document 1653623 if they've subscribed to that particular channel.

PubSub requires a server side implementation (there are many existing open source solutions/frameworks in just about any language), but it doesn't need to check or poll your data source. You'd simply implement publising in your service layer:

POST /document
...
// Update the document
session.save(document);
// Publish the document to it's channel
pubSub.publish("/document/" + document.id, document);
 
This is an interesting problem and would also like to know what solution you would use. I am thinking that your clients could subscribe to a push server or message queue that notifies them of changes made by the other users.
 
debonair said:
Take a look at http://faye.jcoglan.com/
Click to expand...

Depending what language you using the solution to this problem is extremely simple. If its C# you could use an interprocess communication channel.

http://www.codeproject.com/Articles/17606/NET-Interprocess-Communication
Click to expand...

You could use a messaging broker infrastructure such as RabbitMQ (runs on linux, extremely light weight and very fast, i've got it processing around 9,000 messages a second), you can then use MassTransit or the AMQ library. You could also use MSMQ which by default comes installed with pretty much every windows machine.

Alternatively you can even do a UDP broadcast over the network and make your application listen in the background for specific messages.

public void SendPulse<T>()
{
IPEndPoint ipep = new IPEndPoint(IPAddress.Loopback, 10000);
byte[] data = new byte[1024];
using (var udpClientSocket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp))
{
data = Encoding.ASCII.GetBytes(string.Format("{0}#{1}", typeof(T).Name, typeof(T).Assembly.Location));
udpClientSocket.SendTo(data, 0, data.Length, SocketFlags.None, ipep);
}
}
Click to expand...

Usage
Pulse _pulse;
lock (_lock)
{
_pulse.SendPulse<T>();
}
}, () => _logger.Info("Pulse monitor bound to handler"));
Click to expand...


Receiver (clients)

void ReceiveData(IAsyncResult resultAsync)
{
IPEndPoint sender = new IPEndPoint(IPAddress.Any, 0);
EndPoint tempRemoteEP = (EndPoint)sender;

// Get the Socket
Socket remote = (Socket)resultAsync.AsyncState;

// Call EndReceiveFrom to get the received Data
int recv = remote.EndReceiveFrom(resultAsync, ref tempRemoteEP);

// Get the Data from the buffer to a string
string stringData = Encoding.ASCII.GetString(buffer, 0, recv);
string node = string.Empty;
string path = string.Empty;

if (!string.IsNullOrEmpty(stringData))
{
string[] split = stringData.Split('#');
node = split[0];
path = split[1];
}

if (!_nodeHeartBeats.ContainsKey(node))
{
_nodeHeartBeats.TryAdd(node, DateTime.Now.ToUniversalTime());
_nodeSpawnLocations.TryAdd(node, path);
}
else
{
_nodeHeartBeats[node] = DateTime.Now.ToUniversalTime();
}

// Restart receiving
udpServerSocket.BeginReceiveFrom(buffer, 0, 1024, SocketFlags.None, ref ep, new AsyncCallback(ReceiveData), udpServerSocket);

}
Click to expand...


This could be easily adapter for what you need.
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X