Computing desk
< May 19	<< Apr \| May \| Jun >>	May 21 >

Welcome to the Wikipedia Computing Reference Desk Archives
The page you are currently viewing is an archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

May 20

How come a shared library could provide service?

In the previous quesiton, somebody has mentioned a different way to use shared libraries:

... if you write a library as just that - a library of calls that anyone can call, with no system-global state, then there is essentially nothing different to do. But some libraries do decide they need system global state; that's not necessarily a wrong thing to do, but it means the library is performing a service, rather than just being a bunch of code for others to call. If a library wanted to do that, it would be doing IPC (so it might create a shmem (shared memory) block or something) ...

As far as I know services are usually provided by daemons or servers (e.g. crond, sshd, dbus-daemon, XServer, Compiz window manager, httpd, ftpd) which are all running processes. Each server process has its own memory to keep variables, constants and all information (which may be so-called system global state) needed to provide service. And client processes can ask for service from server processes by various IPC mechanisms.

On the other hand, we all know that IPC means Inter-"Process" Communication, which is used to communicate between processes. But a shared library when not executed is just a binary file residing in file system. So how could a client process asking for service communicate with a non-process or a binary file via IPC? Is it really possible? Did I miss something? Misunderstood? Or examples? -- Justin545 (talk) 09:56, 20 May 2013 (UTC)[reply]

Indeed, a library isn't a process and doesn't have its own threads or memory. If a library is used by two processes, those two can do IPC. Global state can be as trivial as the library storing stuff in a regular file. If a more sophisticated service is needed, the library can double-fork off a proper daemon. -- Finlay McWalterჷTalk 11:23, 20 May 2013 (UTC)[reply]

You meant client process calls the library and then the library forks off a new daemon? So the server process is actually created by its client indirectly. Mmm... it sounds pretty interesting...

And you were right. System global state should be kept in shared memory. Regular file may not be ideal for being the place for system global state, as it involves disk IO which is pretty slow (unless the file is mmap()ed). If I don't get it wrong, shared memory block created by shm_open() can persist in the system even if the calling process is terminated. Occasionally, when type command ipcs we can find orphan shared memory block which might be from shm_open().

But may I ask... Why should service always be served by processes? The Linux kernel itself is *NOT* a process anyway, but still it provides almost every services we can imaging very efficiently. -- Justin545 (talk) 12:30, 20 May 2013 (UTC)[reply]

State can also be stored in hardware. For example, if your library is an I/O controller (say, even something as straightforward as a file storage API), then reading and writing data to disk stores state. A graphics library may draw data to an external display, and that state can persist, even if the process that performed the data transaction terminates. A power-control library can start a fan spinning, or turn off an unneeded piece of hardware. The software need not remain active in memory if the hardware is specifically designed to hold state. Nimur (talk) 12:13, 20 May 2013 (UTC)[reply]

"The software need not remain active in memory" exactly! It is the point! As I replied above, shared memory should be able to persist in the system if it is created by shm_open(). If I don't get it wrong, it is allowed for a block of shared memory exists in the system but without any owner process. In which case, there is no any active code or processes running but the system-global state gets preserved! -- Justin545 (talk) 12:55, 20 May 2013 (UTC)[reply]

Storing state in hardware makes me think of kernel modules (.ko) which are also kind of shared libraries, semantically. Kernel modules are neither active code nor processes/threads. Yet they can preserve system global state and provide service to client applications. As you said, the state should be stored in hardware. -- Justin545 (talk) 13:32, 20 May 2013 (UTC)[reply]

Windows DLLs do allow one to define data sections as shared between different processes, Unix type systems normally don't have that sort of facility provided by the linker, one has to do one's own system call to set up a writable shared area. Other systems in the past have also provided similar type facilities but nowadays it is seen as something the user should do for themselves if they really want it and not something to be encouraged in general. Also there always seems to special requirements which a user can handle better themselves. For instance one might want persistent state whereas a system implementation would normally just discard the data in such an area when the last use of the library unloads. Dmcq (talk) 12:31, 20 May 2013 (UTC)[reply]

Actually what's a more common requirement now is for thread-local storage where a number of threads share the same address space. That causes problems for all writable data, for instance errno needs to be thread local. Dmcq (talk) 12:44, 20 May 2013 (UTC)[reply]

It is the reason why I asked the previous question. I was worried that multiple processes using the same shared library would interact with each other somehow and crash. In your Windows DLL case, the processes interact with each other by the shared data sections. Extra concerns should be made to design shared library code. -- Justin545 (talk) 13:03, 20 May 2013 (UTC)[reply]

"For instance one might want persistent state whereas a system implementation would normally just discard the data in such an area when the last use of the library unloads.". Yes, that should be reference counting. Windows often perform reference counting on its kernel objects. File descriptors are also reference counted under Unix-like systems.

So errno is not thread-local? I always think it is before... -- Justin545 (talk) 13:15, 20 May 2013 (UTC)[reply]

It is thread-local, that's what I was saying. If one did nothing special it wouldn't be thread local. Dmcq (talk) 14:09, 20 May 2013 (UTC)[reply]

Ok...sorry about that... -- Justin545 (talk) 14:27, 20 May 2013 (UTC)[reply]

I've never heard of a facility in Windows DLLs for defining data sections that are shared across processes, and I don't see how it could work in a way compatible with the NT security model. -- BenRG 19:43, 21 May 2013 (UTC)

You use a pragma to put data declarations into a magic section called .shared - see this discussion for a modern example (although this has been in win32 a long time, I think since win98 or so). As far as I know, behind the scenes the kernel essentially establishes a shared memory block, as if CreateFileMapping had been used. -- Finlay McWalterჷTalk 20:09, 21 May 2013 (UTC)[reply]

To be clearer, you create a special section, put shared data into it with a compiler pragma, and use a linker option to mark that section RWS:

//#pragma comment(linker, "/SECTION:.shared,RWS")
#pragma data_seg(".shared")
  int foobar=0;
#pragma data_seg()

As far as I know (which means I haven't tried it) it still works in current Windows systems. -- Finlay McWalterჷTalk 20:17, 21 May 2013 (UTC)[reply]

How to open a port under Slackware 14

Hi:

I am just wondering how do I open a port under Slackware 14. My computer is set up as the DMZ computer of the network so all ports are physically open from the router's end of things. The port in question is 1494. I used an open port check tool website and found that 1494 is closed. I issues the "iptables -I INPUT -- tcp -p 1494 -j ACCEPT" command and use the open port check tool website again and found that 1494 is still closed. So I must not be doing the right thing. Any insight is greatly appreciated. Thanks,

76.75.148.30 (talk) 12:18, 20 May 2013 (UTC)[reply]

Try this. Not the actual solution (the guy's router was at fault), the trouble shooting steps. Vespine (talk) 00:50, 21 May 2013 (UTC)[reply]

Non-proprietary GPS database schemas

Are there any? I was just curious how they get from lat-long coordinates to street, but couldn't find any freely viewable GPS database schemas. 20.137.2.50 (talk) 16:26, 20 May 2013 (UTC)[reply]

I don't think so. I just did a search and turned up nothing helpful. Hopefully another user will find something. --Yellow1996 (talk) 18:34, 20 May 2013 (UTC)[reply]

I believe OpenStreetMap uses a freely accessible database, but I think it is much less complete than the ones used by major commercial navigation services (and is most complete for parts of Europe). Looie496 (talk) 19:44, 20 May 2013 (UTC)[reply]

In particular, OSM's Nominatim service does reverse geo lookup. So 37.397444 N by 122.106045 W, when munged into a query string that Nominatim likes, yields this XML data, which has the street address, including house number. -- Finlay McWalterჷTalk 20:19, 20 May 2013 (UTC)[reply]

Gee, that's awfully close to my house! Are you playing at a point, Finlay? Nimur (talk) 14:20, 21 May 2013 (UTC)[reply]

A long time ago, I used to live down on Ortega. Oh how I miss Armadillo Willy's. -- Finlay McWalterჷTalk 16:59, 21 May 2013 (UTC)[reply]

So under the hood do they have utterly huge tables with a column containing every possible lat:long pair within the area mapped? 75.75.42.89 (talk) 23:44, 20 May 2013 (UTC)[reply]

No, they have a giant geographic database of points, vectors, polylines, and polygons, with labels, layers, and other meta information, all tagged with lat/long. Their database engine lets them do queries like "what is the nearest polyline to this coord" or "what are all the objects in the database that fall within this rectangle". -- Finlay McWalterჷTalk 12:07, 21 May 2013 (UTC)[reply]

How would the database engine quickly find the nearest polyline to a given coord, to use your example? Run the ~~pythagorean theorem~~ appropriate distance calculation with the lat/long tags of every polyline in the database and your coords, keeping track of the current nearest and updating with nearers as you go? That would run in O(n) time, but if the database were organized cleverly by the coords, as opposed to the coords just being tags available for each object, I thought something better were being done. 20.137.2.50 (talk) 15:00, 21 May 2013 (UTC)[reply]

What's important is how the indices are organised; as we want to do spacial queries, we want a spatial index. -- Finlay McWalterჷTalk 15:51, 21 May 2013 (UTC)[reply]

R-tree is a good read. Thanks. 20.137.2.50 (talk) 14:21, 22 May 2013 (UTC)[reply]

Is it possible to acceess different parts of a class by different threads?

Hi !my doubts are based on below programme.

    class C implements Runnable  
   {  
   public void f1()  
   {  
   System.out.println("Hi");  
   }  
   public void run()  
   {  
   f1();  
   }  
   synchronized void f2()  
   {  
   System.out.println("Hello");  
   }  
   }  
   class synchronized  
   {  
   public static void main (String args[])  
   {  
   C c1=new C();  
   Thread t1=new Thread(c1);  
   Thread t2=new Thread(c1);  
   t1.start();  
   t2.start();  
   }  
   }

Here c1 is object of class C.
I have accessed method f1 of object c1 by threads t1 and t2.
My Doubt:
I want to access method f1 by thread t1 and f2 by thread t2.
Is it possible?
If possible please tell me
--Me shankara (talk) 17:42, 20 May 2013 (UTC)[reply]

This is the same question you've been asking for over a year, over two accounts. Bluntly, there's no indication that you've paid any attention to the very detailed answers that I and several others have given you. -- Finlay McWalterჷTalk 17:47, 20 May 2013 (UTC)[reply]

(after edit conflict) Not the way you're doing it. You are giving both threads the same object, and the object's run() method will work the same way. It has no way of knowing it should call f1() in one thread and f2() in another. However, you can achieve what you want by creating two C objects, with different parameters in the constructor, and have the run() method call different methods based on them. You'll have to modify your C class, I'll leave the details to you. JIP | Talk 17:50, 20 May 2013 (UTC)[reply]

Thanku Mr JIP --Me shankara (talk) 01:41, 21 May 2013 (UTC) — Preceding unsigned comment added by Me shankara (talk • contribs) 01:40, 21 May 2013 (UTC)[reply]

Fiddling with volume on Windows 7

For some reason, recently, Flash player's volume defaults to low. Very low. To fix it I have to open up the mixer and manually raise Flash's volume every time I load it.

1. Can I reset all of the volumes in the mixer to default? With all the fiddling around I've been doing, a lot of them are on weird volumes, 23, 48 and so on, and getting them back to exactly what they were is fiddly, pixel-perfect work, as far as I can tell.

2. Is there any way to change Flash's default volume?

3. Any idea why this might have happened and what I can do to prevent it from happening again?

94.11.173.214 (talk) 18:38, 20 May 2013 (UTC)[reply]

Unfortunately, it looks like setting the mixer levels back to default isn't possible in Windows 7; according to here, and here. Check out this page for setting the default volume in Flash Player. Hope this helps! --Yellow1996 (talk) 18:53, 20 May 2013 (UTC)[reply]

the md5 sum of an empty string

Why is the md5 sum of an empty string is d41d8cd98f00b204e9800998ecf8427e? Shouldn't it be undefined? OsmanRF34 (talk) 23:35, 20 May 2013 (UTC)[reply]

According to the article MD5#MD5_hashes, there is 0 padding added to the message before the hash is calculated. RudolfRed (talk) 03:53, 21 May 2013 (UTC)[reply]

Almost... RudolfRed is correct that the string is zero-padded; but zero-padding is not the reason that an empty-string has a well-defined hash. If so, an empty string would produce the same md5 hash as a string of just a few zeros. In fact, if zero-padding were the explanation, every set of non-block-sized inputs would always collide with their zero-padded counterparts - for example, we would expect the hash for "\0" and "\0\0" to be identical, as they will both be zero-padded to the block-size. In fact these strings (correctly) yield different md5 hash values. Nimur (talk) 13:57, 21 May 2013 (UTC)[reply]

No, it shouldn't. If it were undefined and you tried to send an empty file, you could not decide what sum append to the file and its recipient would be unable to check if the file was intentionally empty or it was a transmission error. --CiaPan (talk) 05:46, 21 May 2013 (UTC)[reply]

There's no sum appended. Just 0s... (actually this isn't entirely true, the padding is zeroes followed by a length byte... if I remember right). Part of why a zero string goes to a non-zero output is because all of these hashes have constants pre-defined that mix into them. Sort of like s boxes in symmetrical encryption algorithms; I realize that comparison isn't perfect btw. Shadowjams (talk) 19:10, 21 May 2013 (UTC)[reply]

So what's the reason of calculating md5 (or any other check-sum) at all? If you do not add it to the message which contains a file, how could the recipient verify if the file is not corrupt? Whether you transmit data as an e-mail attachment, sell it on CD or publish on an HTTP or FTP server, you need to add the check-sum in some way (in a separate file or as a text part of an e-mail message or the web page). --CiaPan (talk) 05:46, 23 May 2013 (UTC)[reply]

All of these hashes work by having an internal state which is initialized with nothing up my sleeve numbers, mixing blocks of the message into the internal state, then deriving a hash value from the internal state. A 0-length message is padded to one block, but even if it weren't it would still have a well defined hash value, which would simply be the hash derived from the initial state, with no mixing stages.

The padding method is described at MD5#Algorithm. The important thing is that it be reversible, since that guarantees that different messages will not be converted to the same padded message. Padding with zeroes isn't reversible since the message might end with a zero. One way of making it reversible is to append a single 1 bit, then pad the rest with zeroes. Another way is to encode the original length at the end of the padding. MD5 does both, for whatever reason. -- BenRG 19:39, 21 May 2013 (UTC)