advice on Win APIs and libs for project

[padren]

Hey everyone

 

I am working on a new file sharing program for synching with other systems. I am not too happy with FTP and normal windows sharing, so I want to try out an idea to make it easier to update server files while editing them locally.

 

My concept works like this:

 

In the synch program, you setup the prefs and browse to and mark a folder, and enter the info for what server you want it synched with.

 

After that, it DLs the files from the server, mirroring them locally, but also caching the file info that the server supplies seperately. This way, each file locally is associated with info such as the server's file size (since CRLF/CR difs can occur if its linux server and a win client) and the server's last modified date.

 

In the synch program, you can resynch, which will download any file newer than yours, and upload any file you have that is newer than what is on the server, but it will also check for write-collisions, since it will compare the cached server data, and if the server has a file older than your newly modified version, but that happens to be older than the one you downloaded, it will flag that as a conflict.

 

 

About speed:

 

It will have an FTP optional backup incase your server cannot open new ports (such as on shared hosting accounts) but the primary transport method will utilize a maintained open connection between the server and the client.

 

The keep-alive connection is geared to fewer clients with faster response time, and for text files (I am mostly trying to speed up upload of source code, HTML, and other plain text files) it will build a change list based on the line breaks, consisting of "delete", "insert" and "modify" actions.

 

Since it keeps a cached copy locally of the file as it exists on the server, it shouldn't be too hard to ensure the change list will result in an identical file.

Along with the changelist, a checksum of the file is sent, so the server can send an OK response when it does its own checksum, or request a full transfer of the whole file and log the error of the optimizer.

 

 

And since its a constant connection, there is no "connecting to host" lag, but instead something more akin to the response time of a well designed network first person shooter game. (I'll have to play with reliable packets vs tcp a bit to find the most optimized transport options.)

 

 

I already plan to use Raknet for the networking, and I have visual C++ express to write it in (so the server may be linux or windows based) but I would really like some advice on the windows client end of things.

 

 

If possible, I would much prefer to skip the forced manual synch button all together, and hook into some windows API that lets me "hook" callbacks into operations such as "create folder" and "file save" etc.

 

I know some programs do this, and even allow you to "fake" folders and open zips as if they were directories.

 

I have no idea what these APIs are called or what to look into reference material wise. I just want to be sure I can open/edit/save in my favorite editors to a folder like any other local folder, and preferably have the rest of it all work behind the scenes (I'll make the user bind a hotkey to "synch" if I can't hook into the correct windows events).

 

Any advice on the windows event APIs or where I need to look this stuff up? I don't even know what the libs are called.

 

TIA

[Aeternus]

Your idea of keeping things up to date and only uploading and downloading changed versions sound pretty much like the features you get in most version control software (see cvs, svn etc).

[RyanJ]

Here is an idea, if you want to fake a file type, make your own and then assign it the icon you want. This can allow you to "fake" a zip file or anything else.

 

As for global hooks, I suggest you look at MSDN, something like this: http://msdn.microsoft.com/msdnmag/issues/02/10/CuttingEdge/

 

There are loads of others there that explain how you could impliment them

 

Cheers,

 

Ryan Jones

[bascule]

Okay, I'm going to be a bit biased on this, but...

 

Stay away from MFC. They suck. Like everything Microsoft seems to make, they suffer from horrible bloat.

 

Stay away from the Win32 API unless you have a few months to devote to actually learning it "properly". And by "properly", I mean you wrap it up in a C++ wrapper to make it actually usable (this, BTW, is what Microsoft tried to do, and failed, with MFC)

 

If I were starting a Windows development project nowadays I'd be looking at Qt or wxWindows

[encipher]

Okay' date=' I'm going to be a bit biased on this, but...

 

Stay away from MFC. They suck. Like everything Microsoft seems to make, they suffer from horrible bloat.

 

Stay away from the Win32 API unless you have a few months to devote to actually learning it "properly". And by "properly", I mean you wrap it up in a C++ wrapper to make it actually usable (this, BTW, is what Microsoft tried to do, and failed, with MFC)

 

If I were starting a Windows development project nowadays I'd be looking at Qt or wxWindows[/quote']

 

 

I disagree with your statement that padren should stay away from WinAPI. Indeed MFC is crap, but there is no reason to not use winapi. Its simple, dynamic and flexible when it comes to GUI coding. As for wrapping it, its not nessicary since the winAPI was written in C in the first place. A lot of applications i write use little or no C++ and its fine.

 

As for your question about hooking an API to sync. I don't think that is really nessicary. If the user launches an editor from within your software.. you can use CreateProcess to execute the program and then you have full control over what happens. You can wait till the program exits then compare the new file with the old one to see if changes were made etc..

[padren]

Here is an idea' date=' if you want to fake a file type, make your own and then assign it the icon you want. This can allow you to "fake" a zip file or anything else.

[/quote']

The issue though is I need to be able to open normal files (htm, html,cpp,php,cfm,pl,asp,txt,etc) so any third party application editing those files need to be able to identify, browse to and open a folder with those files in it.

 

If there was an abstract folder API independant of the file system (where say, you simply subclassed a base directory class and had your own methods for write,read,createdir,move/delete etc) then it would be doable, but if its possible it may be a pretty tricky API.

I know Allaire did it a long time ago with their RDS setup, but they also ended up scrapping that methodology I think.

 

As for global hooks, I suggest you look at MSDN, something like this: http://msdn.microsoft.com/msdnmag/issues/02/10/CuttingEdge/

 

There are loads of others there that explain how you could impliment them

 

Cheers,

 

Ryan Jones

 

That gives me some good names of the APIs (ReadDirectoryChangesW API, etc) that can give me a good lead in on where to start ferreting out deeper info - thanks.

 

Your idea of keeping things up to date and only uploading and downloading changed versions sound pretty much like the features you get in most version control software (see cvs, svn etc).

 

I don't want to build another batch synch system though, I want exceptional realtime performance that burns decent CPU cycles to make a small number of clients very responsive, which is why I want each connection to the server to stay persistent while working for hours at the office or use datagram packets.

 

I am pretty sure I can get performance up there with a decent first person shooter network game, and when I am saving every 3 seconds while editing code the 1-2 second pause that the windows file sharing causes at random intervals is somewhat annoying, and FTP is like working underwater.

 

I expect I can blast under 200 bytes of data over an open socket (if you save after editing one line, which often happens in debugging) and get the response time down low enough that you'd think you were working on the local network, even when editing a 2000 line source file.

 

 

Okay' date=' I'm going to be a bit biased on this, but...

 

Stay away from MFC. They suck. Like everything Microsoft seems to make, they suffer from horrible bloat.

 

Stay away from the Win32 API unless you have a few months to devote to actually learning it "properly". And by "properly", I mean you wrap it up in a C++ wrapper to make it actually usable (this, BTW, is what Microsoft tried to do, and failed, with MFC)

 

If I were starting a Windows development project nowadays I'd be looking at Qt or wxWindows[/quote']

 

That is fair advice, and I do try to stay away from MFC. I would prefer to use wxWindows, but this app really has no visual components at all. I think I could even handle it all from a config text file if needed, other than starting/stoping etc.

 

The trouble is hooking into file and directory system level events, which I doubt wxWindows is designed to do. (If it can please correct me. )

 

 

As for your question about hooking an API to sync. I don't think that is really nessicary. If the user launches an editor from within your software.. you can use CreateProcess to execute the program and then you have full control over what happens. You can wait till the program exits then compare the new file with the old one to see if changes were made etc..

 

Well, I'd rather my system work completely independant of the third party editors, and I'd hate to close photoshop and reopen it when I need to update a few graphics.

I'd like to keep it decoupled from applications, and work directly with the file system itself for its triggers.

 

 

 

I'll try searching for those windows APIs, as well as any opensource wrappers that may have been written when others were trying to cope with cleaning up MS's messes.

 

Thanks for all the advice, I'll let everyone know how it goes, and if anyone has any use for this sort of tool it'll be opensource itself.

[encipher]

I dont quite understand your need for system wide hooks. Although they may be useful, it should be the last approach when tackling a problem. What exactly do you want to acheive.. there's a whole set of functions that deal with detecting changes in files, etc.. that are free of any hooks (At least none that you deal with).

 

 

FindFirstChangeNotification

 

 

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/findfirstchangenotification.asp

 

regards

 

encipher

[padren]

I dont quite understand your need for system wide hooks. Although they may be useful' date=' it should be the last approach when tackling a problem. What exactly do you want to acheive.. there's a whole set of functions that deal with detecting changes in files, etc.. that are free of any hooks (At least none that you deal with).

 

 

FindFirstChangeNotification

 

 

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/findfirstchangenotification.asp

 

regards

 

encipher[/quote']

 

 

I think that is exactly what I need - I just didn't use the term "hook" correctly.

 

Thanks again