"Remote" Encoding

stroebs

stroebs

Expert Member
Joined
Jan 15, 2009
Messages
2,109
Reaction score
41
Location
Cape Town
Hi peeps,

I am fairly new to the more advanced workings with linux, have only done simple things like downloading torrents, setting up websites, DNS with a BIND server using Webmin and some encoding.

My question is such: Is there a way to use a remote machine to do encoding on? IE Feed the machine a command over the internet (Could be via VPN if necessary) and let it encode something for you? I've seen a few services do this online and wondered if it would be possible to do this sort of setup myself.

Before anyone posts any lmgtfy links, I have tried Googling it but I am either using the incorrect terminology or nobody has really laid this out bare in a public forum before.
 
MyWorld said:
It is fairly straight forward:
Code: 
$ ssh username@ipadress
Enter password

Issue command

If you close a ssh session it kills all instructions from that session, so you need to look into "screen", it should take you 10 minutes to acquaint yourself with it, and off you go!

http://magazine.redhat.com/2007/09/27/a-guide-to-gnu-screen/
Click to expand...

So would I be able to send a command using a file on my local machine?
As a prime example, I am using HandBrakeCLI to encode files, and I want to use a remote machine to do the brute work of the encoding without having to actually transfer the files between servers.
 
stroebs said:
So would I be able to send a command using a file on my local machine?
As a prime example, I am using HandBrakeCLI to encode files, and I want to use a remote machine to do the brute work of the encoding without having to actually transfer the files between servers.
Click to expand...

You will need the files & handbrake cli on the remote machine you ssh into. How do you propose encoding a remote file without physically transferring it to that machine?
ssh gives you secure access to the remote machines shell.
 
It might not work exactly as you are planning, but have a look at:
http://www.encodingcluster.com/trac/

You need to remember that in order to encode a file the client needs to download a section of the video file and encode it and then re-upload it to the server and then the server needs to either "stitch" it together again or use whatever means to join the files again. So there WILL be data transfer between the server and client(s).
Think SETI, it works in this fashion, you get a chunk of data and work through it on your machine, re-upload that chunk and then the server processes it again and put the different chunks together again.

There are not many video encoding solutions for this since very few people have this need outside the movie studios.

So unless you have a cloud farm or are going to do this for like say a living, why not just wait 20 minutes for it to complete on your PC? If it is for a company that have that much encoding to do and they cannot schedule it for when the server is not in use, get a mini server to handle it, like the HP ProLiant MicroServer, cheap and should get the job done.

Here is a discussion on it listing possible Mac, Windows and Linux solutions with varying degrees of success:
http://superuser.com/questions/2718...64-encoding-jobs-across-multiple-computers-to

The only other option is to maybe join the Doom9 forums and hear from them?

Eish I have said way too much... need some tea...
 
MyWorld said:
It might not work exactly as you are planning, but have a look at:
http://www.encodingcluster.com/trac/

You need to remember that in order to encode a file the client needs to download a section of the video file and encode it and then re-upload it to the server and then the server needs to either "stitch" it together again or use whatever means to join the files again. So there WILL be data transfer between the server and client(s).
Think SETI, it works in this fashion, you get a chunk of data and work through it on your machine, re-upload that chunk and then the server processes it again and put the different chunks together again.

There are not many video encoding solutions for this since very few people have this need outside the movie studios.

So unless you have a cloud farm or are going to do this for like say a living, why not just wait 20 minutes for it to complete on your PC? If it is for a company that have that much encoding to do and they cannot schedule it for when the server is not in use, get a mini server to handle it, like the HP ProLiant MicroServer, cheap and should get the job done.

Here is a discussion on it listing possible Mac, Windows and Linux solutions with varying degrees of success:
http://superuser.com/questions/2718...64-encoding-jobs-across-multiple-computers-to

The only other option is to maybe join the Doom9 forums and hear from them?

Eish I have said way too much... need some tea...
Click to expand...

This is an excellent reply, thank you.

I think I will end up transferring the data to a local machine, encoding and uploading again.

Main reason i wanted to do this is so I could do a queue - But I guess even then i could write a script which would transfer the file to the remote (or local) machine, run the encoding job through ssh and then pull it back again with the new encode. Also my remote server only does ~6fps on a good day and the machine I have here does ~40fps.
 
Sticking to the SSH idea, you could also use the SSH File system (SSHFS) and remotely mount the local PC folder where you keep the files into the remote PC so the remote PC works on files as if it were local but the files always remain on your PC.

Not sure how much bandwidth this will use. Will probably slow down performance of encoding as it has to fetch bits of the files over the internet.
 
nic777 said:
Sticking to the SSH idea, you could also use the SSH File system (SSHFS) and remotely mount the local PC folder where you keep the files into the remote PC so the remote PC works on files as if it were local but the files always remain on your PC.

Not sure how much bandwidth this will use. Will probably slow down performance of encoding as it has to fetch bits of the files over the internet.
Click to expand...

As for this, I think it'll still be faster than encoding at 4-6fps!

Running on a Neotel 4Mbps fibre I get a sustained ~400KB/s no matter what time of day it is, which will make it encode at around two to three times the speed of my current encodes.

I'll look into the SSHFS option sometime today.
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X