libCurl + tinyXML / lock file

[spacegaier]

Hello toghether,

I am currently searching for a solution to the following situation: I have a XML file lying somewhere in the internet. I want to access it in order to give it's content to tinyXML to be parsed. I tested it with libCurl and it works so far.

However, I also want to reach the following: If libCurl read the file, I want it to be locked until I release it as my application will later manipulate te XML file and while doing so no other client of my programm should be able to also access the file. How would I realize something like this? libCurl does not really support that as far as I can tell (although there is some kind of parameter CURLSHOPT_LOCKFUNC, but I was not able to get any sense in it).

Anyone having ideas / experience on how to solve that?

PS: Should I go for other libraries to solve this?

[aguru]

I have a XML file lying somewhere in the internet.

What exactly do you mean by that? It seems you need write access for what you want to do, ie something like ftp?

[stoneCold]

Placing the XML in a database would be my solution, at least if it should be used for something serious.
That way you can take care about concurrency and coherence much easier than with plain file access.

Just my 0.02$

[volca]

Any way will involve some kind of locking mechanism. Http server won't provide such thing itself, so you could as well use CVS or SVN, WebDAV, custom server with a custom protocol...

[stoneCold]

For example, I know that with MySQL it's possible to Lock/Unlock tables out of the box, so this should ease the task, that was just the thing that I wanted to point out.
I for one, would prefer to use MySQL over some more involved systems like CVS, SVN, etc by far.

[volca]

The problem with xml in SQL database is that the databases are not exactly designed to store big amounts of data per row. Sure you have the memo field type, but still, storing data of more than a certain size can cause problems. WebDAV is probably the most fitting for the type of functionality the OP had in mind (And it provides file read, write and locking operations).

[stoneCold]

The problem with xml in SQL database is that the databases are not exactly designed to store big amounts of data per row.

Agreed, storing a large XML string inside a database row would be a bad idea of course, but it should be possible to use a more clever approach for storing the XML inside the database.

WebDAV is probably the most fitting for the type of functionality the OP had in mind (And it provides file read, write and locking operations).

Didn't know WebDAV, but it looks quite fitting for your needs, spacegaier.

The protocol consists of a set of new methods and headers for use in HTTP and is almost certainly the first protocol ever to use XML[citation needed]. The added methods include:

• PROPFIND — Used to retrieve properties, stored as XML, from a resource. It is also overloaded to allow one to retrieve the collection structure (a.k.a. directory hierarchy) of a remote system.
• PROPPATCH — Used to change and delete multiple properties on a resource in a single atomic act.
• MKCOL — Used to create collections (a.k.a. directory).
• COPY — Used to copy a resource from one URI to another.
• MOVE — Used to move a resource from one URI to another.
• LOCK — Used to put a lock on a resource. WebDAV supports both shared and exclusive locks.
• UNLOCK — To remove a lock from a resource.

Cheers

[spacegaier]

I did not want to go for a database (although I this would certainly fit my internal application data/logic structure) as this would be an extra hassel for the user. By using a single (big) XML file, they only need a bit of webspace and that's it.

I did not know WebDav either but it looks quite promissing. Will try it. libCurl does not support it though, so I will have to switch to neon.

Thanks for the hints. I'll let you know, if anything new occures...

EDIT: Ahh wait. Do I need a special server for using WebDav? In this case, this solution would also be gone .

[CABAListic]

You need a server which handles the WebDAV extension, yes. The Apache server definitely does, but I don't know how many webspace providers enable it for their customers...
However, a plain HTTP server will simply not work, there is no way to lock a file via the HTTP protocol. Besides, how are you going to update the contents?

[spacegaier]

Besides, how are you going to update the contents?

No real ideas yet. Still in the discovering phase and checking if and how this could be done.

And there is still the "problem" with the locking. Will have to investigate the spreading of WebDav (and keeping my eyes open for potential different solutions).

I could still, create a dummy file next to my actual content file when the content is used and therefore forbid reading and manipulating it, but that would be an awkward solution.

[CABAListic]

My point is, you first need to find a solution where you can actually write content before you can even worry about locking a file. This rules out a simple HTTP server. FTP access would work, although there is probably no direct locking support available, either.

Maybe you should elaborate more on your actual scenario, then we might be able to make better suggestions

[spacegaier]

Maybe you should elaborate more on your actual scenario, then we might be able to make better suggestions

What do you mean by this?

I want to write a program that stores all its data on the web (thought of an XML like format), so that all the clients can access and process it from any location. I don't want to use a database if possible as this would complicate the whole set up process. However, this solution would probalby work quite well in regards to functionality and stability.
At startup the client should load the whole data from the web, in order to show it to the user and let them manipulate it. While this reading process (as well as on the writing processes later), no other client should be able to perform actions on the data. After having done the needed actions at the client-side, the content of the file will be updated on the web, by replacing/deleting/extending the content of the file on the web.

[CABAListic]

Well, what is it you are trying to accomplish? For instance, why do you users need to set up a remote data storage? Why do you need concurrent access to that remote storage? Etc.

Edit: Ok, well, I think you basically have three options: WebDAV with filelocking, a database server, or a custom server. You'll have to choose one The custom server could also be a sort of PHP script which can do the file locking on the server (although I guess such file functions might not be available on all web servers) or use a MySQL database.

[spacegaier]

Well, what is it you are trying to accomplish? For instance, why do you users need to set up a remote data storage? Why do you need concurrent access to that remote storage? Etc.

Well, I need the remote part as the users being in different locations spread over the world need to access the same data. Concerning the concurrent access: The actions one client does, can be logic-breaking to the data another client would get when downloading the data at the same time. Therefore, I need some kind of locking system, to allow one client to extend the data and block others from reading this unstable data state as long as this manipulating process is in action.

Did I make it understandable of do I have to give you more information?

[CABAListic]

No, that will suffice, but as I said, serialising the concurrent access can only be done at the storage place, so you need some sort of backend there which can handle this. Neither HTTP nor FTP will do.

[syd]

serialising the concurrent access can only be done at the storage place, so you need some sort of backend there which can handle this. Neither HTTP nor FTP will do.

boost::asio seems a good candidate for this

for a simple and reliable solution, I'd look at versioning systems like subversion or git.