Backgrounding shell with delay

Syslogd is using udp sockets, unix domain sockets or directly from kernel printf api’s… for reading and writing sockets you only need an file descriptor. A file discriptor doesn’t mean it’s local or nor does it’s number. The number is just an item/index of an array of open files, as an developer I’m glad that there is one file descriptor for all kind of files and connections.

Also here kicks the power of syslog in, you can have one syslogd server gathering information from thousands of other servers. This way you can monitor multiple servers on one machine instead of hanging around all day in an data center. Also with help of audit (since Snow Leopard standard, available since Panther) will get more information send to the syslog daemon.

So saying that you should avoid sockets in syslog system is like saying that you should avoid folders in the Finder. It’s designed for it so you should use it. I even doubt that syslogd is able to work with normal files.

Note: All of the NSApplications (cocoa applications) is using the audit control file (this is an system wide configuration file) that will be read first which is file descriptor 3 on my machine for cocoa applications. Non-cocoa applications have an random file open on file descriptor 3.

Hello.

I think the whole idea of using filedescriptor 3, which is globally assigned, when you have the syslog daemon up and running, is to have some quick and dirty logging facility while you are debugging and testing code.
(It is easier to write to file descriptor three, than enabling loggging.)

Nice to know about audit control, that is an area that has slipped me by, since I seldom get my head over CoreFoundation. :slight_smile:

There are a couple of other auditing tools as well, as core dumps and crash reports, and accton, which is process accounting. the coredumps should really end up in /cores, but usually they end up in my working directory. :slight_smile: CrashReports ends up in ~/Library/CrashReports. See man -s5 core for coredumps, and man -s5 sa for a pointer to other manual pages of accounting, like accton(8). You can double click on the CrashReports, or just look at them in Console.app.

By the way, here is a fine article by John Gruber, about CrashReporter.

Yes, and that you can write your own custom messages instead of using the syslog format.

:slight_smile: I am almost starting to feel morally obligated to find that out, you know, you can just as easy write custom functions for the normal syslog. I like that format, or have learned to live witht it. I use logrotate too, as a filter, which reports anything I want to inspect by localmail. If you don’t follow the syslog format, then I guess it won’t be so informative to have the messages pop up into the Console.app, as Console parses the output.

I also have a library floating around written for turbo-C and MsC, that uses the same principle, but from code, where you can turn on and off logmessages after severity level, by variables and defines.

I won’t keep you posted! :wink:

Because I don’t see me doing that, before it is really necessary, and I don’t value such techniques that involves a configuration manager and syslogd, for other than large server projects, or when I see it hindsightly as feasible! :slight_smile:

I find that sending some pointer values to stderr, in combination with liberal use of gdb, as a far more efficient approach. I’m a strong believer in module testing, when that is possible. And then your code reads better afterwards, without having to run it through cpp to get rid of macro definitions to make it readable. :slight_smile:

You’re missing an important point here, syslogd, crash reporter and audit are not design for developers. It’s the developer’s task to deliver good software, in which they seem to fail over and over again, and all these utilities is for system administrators and end users to see and know what’s going wrong inside the software they’ve bought. It’s not debugging!

I disagree, or think that you have missed a point, the point is to use whatever tool you find feasible, in order to make that program flawless, accounting, can for instance reveal, heavy system load, due to for instance mdfind updating the database while your tool where running. usage of syslog, may let your program run and run, with logging of what is going on, which may help you track those errors. And crashreports, is almost the same as a coredump, that helps you set a breakpoint in gdb, as it shows you where your program failed, if you compiled with debug information.

Not everything can be tracked down with valgrind, leaks and the like.

So, those tools/resources, like Dtrace, serves multiple purposes.

I’m not saying you can’t use it, it’s just not designed to be used by developers because they have their own debugging tools. If those tools were designed for debugging then you wouldn’t have included this in the release software.

The crashreporter data will never be send to the developer, the syslog data will never be send to the developer and audit information also won’t be send to the developer, the developer will never know from these utilities how well their software is running on other machines. While gdb gives an direct feedback to the developer how well it’s software is running, that’s what I call an debugger.

Hello.

My context was that the facilities was to be used on your local developer/test machine, not spreading out to users, I do understand the necessity of sending crash reports, when programs urge you to, since this is a very complex world, and the action of sending in logfiles, makes the stuff I use better.

It was also about using the tools you have available, when there is a point in using them. The account facility for instance, though awkward to use, because of how the metrics are gathered, are still usable, if you are making something that are running all the time.

Well, It suddenly struck me, that I could just do like below, and forget all about the logfile, with routines that opens a logfile, “the usual place”, no need for filehandle 3 nor syslog. :slight_smile:

You are supposed to make your own header file, write formatted strings with fprintf, using this handle,
and close the file when it is necessary. The file is opened for appending text to it.

[code]/***********************************************************************

Name: logerror_file.c   
Created: 04-25-2013
Author:  McUsr with thanks to DJ Bazzie Wazzie.

Usage:   
Return a FILE *handle to a logfile residing in ~/Library/Logs.
The log file should end with a .log suffix. So that it will
be opened in Console.app. Mac Os X.

NB! You have to flush, and use write to write unbuffered 
in connection with signals.

DISCLAIMER
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

***********************************************************************/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
FILE *lfile = NULL ;
static char logfname[FILENAME_MAX] ;

FILE *init_logfile(const char *logfname) ;

FILE *reopen_logfile(void) ;

int main( int argc, char *argv[] )
{
lfile=init_logfile(“logwritertester.log”) ;
if (lfile != NULL ) {
printf(“success\n”) ;
}
return 0;
}

FILE *reopen_logfile(void) {

if ( logfname[0] == '\0' ) {
	fprintf(stderr,"%s: Missing logfilename\n",__PRETTY_FUNCTION__);
	exit(2);
}

if ((lfile=fopen(logfname,"a") )!= NULL ) {
	return lfile ;
} else
	perror("reopen_logfile: Couldn't reopen logfile: ");
return lfile ;

}

FILE *init_logfile(const char *logname)
{
strcat(logfname,“/Users/”) ;
strcat(logfname,getlogin()) ;
strcat(logfname,“/Library/Logs/”) ;
strcat(logfname,logname) ;

if ((lfile=fopen(logfname,"a") )!= NULL ) {
	return lfile ;
} else {
	perror("init_logfile: Couldn't open logfile: ");
}
return NULL ;

}[/code]

Hello.

In all fairness, I was looking into /dev/fd, and I do have a filedescriptor 3 there, or something that poses like one, “I get bad file descriptor” when I use ls on it, and when I use fstat on, both as a user, and as root.

Now, I have some vague recollections of making a filedescriptor in there, as an experiment, maybe that was this one, or maybe it wasn’t, (this experiment happened a long time after I installed syslogd and logrotate.)

But as far as I am concerned, the questions as to which filedescript syslogd grabs, and whether it would grab that filedescriptor on a global basis is still open.

Let me just add that it would be some logic connected to it, if it did, and if a daeom or agent was there first grabbing a file descriptor, that wouldn’t really pose any problems for syslog if it really wanted to grab it, even if that would involve sending a -HUP to that daemon or agent.

That doesn’t mean it is in use by syslog. I would say look at the results of lsof and look for file descriptor 3 of every process. You’ll notice that some processes don’t even us an file on position 3 and I’m having syslogd up and running too :D.

Syslog is shipped with Mac OS X since 10.4 so you don’t need to install it manually. Syslogd is already running after a clean installation of Mac OS X. logrotate doesn’t exists, I think you mean newsyslog which is equivalent to logrotate on Linux.

Define global. Global as in system wide it is not, I thought I was clear about that. Global as within the process yes, all file descriptors are. It is not that syslogd connects to your process to communicate, the syslog client opens an socket (any file descriptor) and communicates from syslog through the socket with syslogd. Like I said before, it’s like a MySQL client connects with a MySQL server. So if file descriptor 3 or 10 is the file descriptor for that syslog socket, it doesn’t really matter.

Some developers like to have a certain file as file descriptor 3 so they don’t have to define an global in their process but they can can just read file descriptor 3.

I did that before my previous post, I had no filedescriptor 3, and syslog wasn’t running.

I actually do mean logrotate, which I have installed on my system.

On the Stack Overflow page referred to earlier, where I discovered this, they did seem to agree on that syslog, had indeed a file descriptor three open, that may very well, be an abastraction for the socket, to make it easier, to communicate with it than put_msg, or whatever, (datagram, init or unix socket).

Those filedescriptors defined under /dev/fd are global system wide.

Such a global filedescriptor 3 is what I do think syslog provides, but I am unable to check that at the moment, but I assure you, I’ll have something up and running in the first little window of time, so that I can get to a conclusion, whether that presumably bad file descriptor 3 of mine, is something broken, or if it is in usage by syslog. :slight_smile:

Hello.

I am having it running, I have no filedescriptor 3 open, I 'won’t try to write to that file descriptor directly, as long as it is bad, (think if I overwrote “/” :slight_smile: ). It also seems like the code in Stack Overflow, just used the file descriptor for reading back the log. The code for syslog, is properietary Apple and is not available at opensource.apple.com, so I guess I’ll never know. :slight_smile:

So, to conclude this: I may have a bad filedescriptor 3, and if syslog opens a global descripter, and whether that file descriptor becomes nr 3, is something I don’t know for sure.

Maybe one day I’ll find out, but in the mean time, your version seems to correlate better with the truth! :slight_smile:

Actually:

I am sorry for being stubborn here, I read later down the stack overflow post, that syslog doesn’t necessarily bind to fd 3, and that I also can see what file descriptors it reads by firing it up manually. So Sorry!

When an process get’s alive an process table is assigned to that process, for each process, with all sort of information. When I open an file, I make an system call to the kernel. the kernel opens the file for me and give me an index back to teh fiel in the file descriptor table. When you need to access the file you’ll send the file descriptor to the kernel and the kernel will handle the file operation for you and return the right data, not the process itself. The process can’t read and write files directly nor can it access the file descriptor table that has been controlled by the kernel for that process.

So in short /dev/fd is not global but refers to the file descriptors for your process so it’s local.

edit: To be sure I was right I read the manual of mount_fdesc and found this line: "fd is a directory whose contents appear as a list of numbered files which correspond to the open files of the process reading the directory. ". Again the file descriptors of /dev/fd seems to only belong to the process that reads them; not globally.

Hello.

I actually meant something like that when I said it was global to the system. That may sound odd, but what I meant, was that the filedescriptors in /dev/fd never will be forked out, when you open a file, or dup a file or whatever. What I should have said is that the file descriptor table is global, so if say you get a file descriptor #4 in one process, that file descriptor appears as taken in a global filetable, so that when another process sends a request for the next free file descriptor through a system call of some sorts, then the kernel will fork back fd #5. That is at least how I understand it.

By the way, there is a nice little app out there that are named Desktop Log, that can display a log onto your Desktop, hopefully it still works on Mountain Lion.

Almost… these file tables (file descriptor table) are created when a process is launched. So every process has it’s own private file table even forked or spawned processes. What I mean is that file descriptor #4 in process 1 doesn’t have to be the same as file descriptor #4 in process 2. Also the first available file descriptor in process 1 isn’t the same as the first available file descriptor in process 2.

How’s that last thing with AppleScript? Because AppleScript isn’t an process on it’s own (it uses sockets through the system) it is possible that two different processes uses the same process to open an file. Which means they share the same file descriptor table because the actual call to the kernel is made by the same process.The file descriptor table that will be used is the file descriptor table of the process that makes the call to the kernel.

I’ve never tested it but I’m sitting here talking with an Linux guru next to me and he said it can be possible with Linux and an proc file system using /proc/PID/fd/n where you’re able to use and locate the file descriptor table of other processes. You have to be root or at least the same user as the process you want to read files from (like stdout).

Hello.

I checked it up, and come to the same conclusion as you did. The file descriptor tables are local to a process, the file table, and the file status table isn’t.

Interesting to know that Applescript uses sockets, and that the file descriptor table stays with that process that opened the file via a socket. I’ll look for that process. :slight_smile:

I figure it to be some job to find out which open file in which process has which filedescriptor. :slight_smile: It seems to me that you would then have to perform at least one, maybe two backwards lookups, from the v-node table.

The last thing you said, would be alot like the program for “cascading stdio”, you provided a link for in another thread. Then it would be like on linux and the proc file system, that you have set it up all front.

You can of course also use /dev/pty, but then you have also rigged it up front.

P.S
Making a more secure version of the program that forces the output over to another terminal, is on the todo list, I’ll message you one day when it is ready. I’ll basically check if the realid isn’t root, and only allow it to run then, and at the same time set the effective id to the real id. If I can make that to work

Yes, I was surprised when I discovered that a few years ago but also makes totally sense at the same time (after I’ve read the anatomy of Apple Events). To be exactly correct, AppleScript uses machports which are similar to unix domain sockets. It makes it easier to understand how the eepc protocol (remote events) works. Instead sending all Apple Events through a local IPC, all Apple Events are send through an network IPC, when the target is an remote machine. It needs to be turned on (in sharing of system preferences) on the target machine to create a network listener for Apple Events, then the target system is able to receive remote Apple Events. After all it’s obvious, but like I said I was surprised at first.

Hello.

I have updated the pick script in post #9 to a command line tool with some c-code from Lucent technolgies, and some of my own. it can be found here

Enjoy.

Edit

I have just updated it, so that you get the previous question repeated if you suspend, (when you can, with ^Z).

You use it in commandlines like cp `pick *` destdir
what it really does, is echoing the arguments you have accepted, and it does so interactively, so if you have answered ‘y’ to something, then you can’t interrupt that, by pressing ‘q’ ^D ^\ or anything else.

You can also execute shell commands from within it, like cat filename
by pressing ‘!’ to get a “command prompt”, if you are unsure if you want to accept the question.

if you givie it a ‘-’ as argument and redirects a file into it, it will prompt for y on every line of the file.

[code]10018$ cat >names
vera
sophie
claudia
ann
mary

10019$ echo pick - <names
vera? y
sophie? y
claudia? y
ann? y
mary? y
vera sophie claudia ann mary[/code]
Any faults are mine, and there are no warranties about what so ever.

Hello.

While I am at it. :slight_smile: Here is a link to idiff an interactive command-line merge tool for merging the differences of two files into diff.out in your current directory. idiff -h
for help.

PS.

There are options for specifying the editor of your choice (works with BBEdit) and passing options to diff.

Enjoy