5 Development Environment Notes

This chapter contains notes about issues and known problems with the development environment software and, whenever possible, provides solutions or workarounds to those problems. The following topics are discussed:

5.1 General Programming

The following notes apply to general programming.

5.1.1 Setting Break Points with dbx and ladebug

The dbx and ladebug debuggers are not able to stop at a breakpoint inside of the _exit() routine. If you attempt to set a breakpoint in this routine, the debugger will continue past it without stopping. Should you need to stop your application just before it exits, try setting a breakpoint in exit() instead of _exit(). Another workaround is to use the dbx debugger with the -noproc switch or the ladebug debugger with the -ptrace switch. For example:

% dbx -noproc myapp

% ladebug -ptrace myapp

5.2 Realtime Programming

The following notes apply to realtime programming.

5.2.1 SA_SIGINFO Not Visible Under Certain Namespace Conditions

The symbol SA_SIGINFO, defined in sys/signal.h, is not visible under certain namespace conditions when _POSIX_C_SOURCE is explicitly defined in the application or on the compile line.

The SA_SIGINFO symbol is visible if you do not explicitly define _POSIX_C_SOURCE. For most applications, unistd.h provides the standards definitions needed, including _POSIX_C_SOURCE. As a general rule, avoid explicitly defining standards macros in your application or on the compile line. If you do explicitly define _POSIX_C_SOURCE, then SA_SIGINFO is visible if you also explicitly define _OSF_SOURCE.

5.2.2 POSIX 1003.1b Synchronized I/O and File Truncation

POSIX 1003.1b synchronized I/O using file status flags does not apply to file truncation. When file status flags are used to control I/O synchronization, no synchronization occurs for file truncation operations.

You can use the fsync() or fdatasync() function to explicitly synchronize truncation operations.

5.2.3 The fcntl() Function and F_GETFL with O_DSYNC File Status

A problem occurs when fcntl() is called with the F_GETFL request, and the file operated on has the O_DSYNC file status flag set. The return mask incorrectly indicates O_SYNC instead of O_DSYNC.

5.3 DECthreads (pthreads)

The following notes apply to DECthreads. See Section 8.10 and Section 8.11 for information about DECthreads interfaces that will be retired in a future release.

5.3.1 Static Libraries

Users who desire optimal performance from DECthreads, and who are willing to relink on future versions of DIGITAL UNIX, may want to use the DECthreads static libraries that are located in the CMPDEVENH425 subset. Once this subset is installed, you can find the libraries in the /usr/opt/alt/usr/lib/threads directory.

Before using these static libraries, you should read the README file in the same location.

5.3.2 Forking

In this release, the metering capabilities of DECthreads may not be reliable in a process that forks.

5.3.3 Signal Handling

Signal handling in the POSIX 1003.1c (pthread) interface of DECthreads is substantially different than signal handling is for the draft 4 POSIX and the CMA interfaces of DECthreads. When migrating your application from the draft 4 POSIX or CMA interfaces to the POSIX 1003.1c interface, please see the IEEE POSIX 1003.1c standard or the Guide to DECthreads for a discussion of signal handling in threaded applications.

5.3.4 Scheduling Behavior (Contention Scope)

In releases of DIGITAL UNIX prior to Version 4.0, thread scheduling attributes were systemwide. In other words, threads had a system contention scope. In Version 4.0 and higher, thread policies and priorities are, by default, local to the process. No artificial limit exists for thread priorities of these process contention scope threads, the full priority range is accessible by every thread.

Previously, there was no way to control the contention scope of a thread. Starting with Version 4.0D, applications coded to the POSIX 1003.1c pthreads interface can set the desired contention scope upon thread creation. For more information on setting and determining thread contention scope, see the descriptions of the following routines in the Guide to DECthreads:

pthread_attr_setscope()
pthread_attr_getscope()

The guide also describes a problem with inheritance of the contention scope scheduling attribute in DIGITAL UNIX 4.0D.

Process contention scope threads provide faster context switches between threads in the same process, and reduce the demand on system resources without reducing execution concurrency. The DIGITAL UNIX "two level scheduling" implementation (the code that supports process contention scope scheduling) automatically replaces kernel execution entities when a process contention scope thread blocks in the kernel for any reason, and it provides time-slicing of compute-bound threads. Therefore, there is no need to worry that using process contention scope will reduce parallelism or allow the execution of some threads to prevent other threads from executing.

The only code that should require system contention scope is code that must run on a specific processor via binding and code that must be directly scheduled by the DIGITAL UNIX kernel against threads in other processes; particularly threads running inside the kernel. While the scheduling policy and priority of process contention scope threads is virtual and affects scheduling only against other threads within the process, the scheduling policy and priority of system contention scope threads (when the process runs with root access) can allow the thread to preempt threads within the kernel. While this can sometimes be valuable and even essential, extreme care must be used in such programs to avoid locking up the system. It may be impossible to interrupt such a thread.

5.3.5 Known DECthreads Problems

The following notes discuss problems with the current implementation of DECthreads.

5.3.5.1 Contention Scope Inheritance

The POSIX standard specifies that the inheritsched thread creation attribute (pthread_attr_setinheritsched) controls the determination of three thread scheduling attributes: the policy, scheduling parameters (priority), and contention scope. When the inheritsched attribute is set to PTHREAD_INHERIT_SCHED (the default), all three scheduling attributes are inherited from the creating thread. When inheritsched is set to PTHREAD_EXPLICIT_SCHED, all three scheduling attributes are set from the attributes object (pthread_attr_setschedpolicy, pthread_attr_setschedparam, and pthread_attr_setscope).

The initial implementation of contention scope support in DIGITAL UNIX Version 4.0D contains an error that was found too late to fix for the release. The contention scope attribute is not inherited, and is always set from the attributes object. Because the default contention scope is PTHREAD_SCOPE_PROCESS (process contention scope), all threads will be created in process contention scope unless the attribute is explicitly set.

A partial workaround is to set the contention scope to PTHREAD_SCOPE_SYSTEM when you want to inherit the system contention scope, but leave the inheritsched attribute set to PTHREAD_INHERIT_SCHED. This causes the scheduling policy and priority to be inherited as usual and results in the creation of a thread of the desired contention scope. Also, when the bug is fixed (in a patch to Version 4.0D and in later releases) the same code will correctly inherit all scheduling attributes, ignoring the explicit contention scope setting (because inheritsched is set to PTHREAD_INHERIT_SCHED rather than PTHREAD_EXPLICIT_SCHED).

5.3.5.2 Freeing User-Allocated Stacks

If you choose to use the new stackaddr thread creation attribute that allows you to allocate your own stack for a thread, beware that you cannot deallocate the stack immediately when a thread returns from pthread_join called with the thread ID of the thread using the stack. That is, in the following code segment, it is not safe to return the memory-mapped stack after pthread_join returns:

pthread_t       thread;
pthread_attr_t  attr;
void            *stack;

 

stack = mmap (NULL, PTHREAD_STACK_MIN, PROT_WRITE|PROT_READ,
MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
pthread_attr_create (&attr);
pthread_attr_setstackaddr (&attr, stack);
pthread_create (&thread, &attr, routine, NULL);
pthread_join (thread, NULL);
/* The following call may cause an access fault in "thread",
because it may still be using the stack to complete
termination. */
munmap (stack, PTHREAD_STACK_MIN);

The problem is that, in technical contradiction to the POSIX standard, pthread_join may return before the target thread has completely terminated. This was not a problem before the implementation of the stackaddr attribute, because although the thread might have existed, it could not have accessed any program-visible resources. Now, because it may still be executing on its stack, the program cannot safely free that stack. There is currently no reliable way to determine when it is possible to free the stack.

This problem will be fixed in a patch to Version 4.0D and in future releases.

DIGITAL does not recommend using the stackaddr attribute. The semantics of this attribute are poorly defined by POSIX and the Single UNIX Specification, Version 2, and as a result, code using the attribute is unlikely to be portable between implementations. The attribute is difficult to use reliably, as the developer must, by intimate knowledge of the machine architecture and implementation, know the correct address to specify relative to the allocated stack. The implementation cannot diagnose an incorrect value because the interface does not provide sufficient information. Using an incorrect value may result in program failure, possibly in obscure ways.

5.4 Analysis Tool with Object Modification

The following notes apply to the Analysis Tool with Object Modification (ATOM) utility.

5.4.1 Using pixie on Applications Built with -om

If your application was built with the -om switch, you must use the previous version of pixie for performance analysis work on the application. This version is located in /usr/opt/obsolete/usr/bin/pixie.

The ATOM-based tools, including the latest version of pixie, cannot currently process executables produced with the -om switch. This is a limitation with the om utility that will be corrected in a future release.

5.4.2 Profiling Signal Handlers and fork with Multithreaded hiprof

A multithreaded program instrumented with the default hiprof command, atom appl_prog-tool hiprof -env threads will sometimes deadlock (enter an infinite polling and sleeping loop). This can happen if the program calls the signal, sigaction, or fork functions or if the program was compiled with a cc -speculate command (which introduces signal handlers).

You can avoid this problem by adding the -toolargs=-calltime or the -toolargs='-calltime -threads' option to the command line. Or, if you must use the default hiprof command for a multithreaded program that calls signal or sigaction, and if libc.so must be profiled, add -exc __sigtramp -exc __Esigtramp macros to the -toolargs option used to exclude the signal-handling procedures and any other procedures they call.

If the program is compiled with a cc -speculate command, omit -all or add -excobj libc.so to prevent libc.so from being instrumented. Similarly, if the application's signal handlers call many other procedures directly or indirectly (for example, in libc.so), you can exclude the whole library instead of excluding a long list of individual procedures.

5.4.3 ATOM third Tool May Not Work Correctly with C++

The ATOM third (Third Degree) tool may not work correctly with C++ applications that use the tasking library (libtask.so or libtask.a). The problem causes ATOM to issue the following warning several times:

atom: Warning: Object 't' has invalid instruction at 0xhexaddress, treating as data.

These warnings are followed by a Third Degree assertion failure:

Assertion failed: pc==InstPC(inst)

You can avoid the problem by linking your application with the -call_shared switch and passing the -excobj libtask.so switch to ATOM. For example:

% atom -tool third -excobj libtask.so myapp

This switch excludes the tasking library from Third Degree's error checking, but avoids the assertion failure.

5.4.4 ATOM Error with Dynamically Linked Fortran Applications

If a dynamically linked Fortran application is instrumented with -all or -incobj libc.so, you may get a loader error similar to the following when running the instrumented application:

pid:myapp.pixie: /sbin/loader: Fatal Error: Multiple instances of
shared object libc.so loaded as both /usr/shlib/libc_r.so and
./libc.so.myapp.pixie

You can avoid this problem by doing one of the following:

If you have the ability to rebuild your application, you can relink it using the -non_shared switch. This produces an application that does not use shared libraries, thereby avoiding the problem.
If you must use shared libraries but do not need to instrument libc.so, you can avoid the problem by passing the -excobj libc.so and -excobj libc_r.so switches to ATOM. For example:
% atom -tool pixie -all -excobj libc.so -excobj libc_r.so myapp

Note that this does not work if you use the Third Degree tool because Third Degree needs to instrument libc.so.
If you need to instrument libc.so (if you are using the Third Degree tool), you can avoid the problem by passing the -incobj libc_r.so switch to ATOM to ensure that both libc.so and libc_r.so are instrumented. After running ATOM, remove the instrumented version of libc_r.so and replace it with a hard link to the instrumented version of libc.so. Then, run the instrumented application. The following example demonstrates the workaround when using the pixie tool:
% atom -tool pixie -incobj libc.so -incobj libc_r.so myapp
% rm libc_r.so.myapp.pixie
% ln libc.so.myapp.pixie libc_r.so.myapp.pixie
% setenv LD_LIBRARY_PATH .
% ./myapp.pixie

5.4.5 ATOM Tools Do Not Support pthread Cancellation

In all configurations, programs that use the pthread_cancel service to cancel a thread of the default PTHREAD_CANCEL_DEFERRED type tend to abort when they have been instrumented with one of the following ATOM-based tools:

third
hiprof if the -calltime, -cputime, -pagefault, or -sigdump options are selected, or if a thread is cancelled after calling the exit() function.
pixie if the -sigdump option is selected, or if a thread is cancelled after calling the exit function.

While the hiprof and pixie tools (without options) tend not to abort with the PTHREAD_CANCEL_DEFERRED type cancellation, all ATOM tools will tend to abort with the PTHREAD_CANCEL_ASYNCHRONOUS type cancellation, as defined by the pthread_setcanceltype routine.

The characteristic symptom is the following error message:

exception system: exiting dues to multiple internal errors:
exception dispatch or unwind stuck in infinite loop
exception dispatch or unwind stuck in infinite loop

As a workaround, you can avoid cancelling threads or use different profiling tools. Other profiling tools include the -p and -pg options of the cc compiler and the uprofile utility. For more information, see the atom(1), pthread_setcanceltype(3), cc(1), and uprofile(1) reference pages.

5.4.6 ATOM Routines Should Not Call Certain Libraries

Section 9.2.5 of the Programmer's Guide, "Writing Analysis Procedures," incorrectly states that an analysis routine can use any system call or library function. Some library routines cannot be called safely from analysis routines in all system configurations.

Standard C Library (libc.a) routines, including system calls, and Math Library (libm.a) routines can be called, except for:

unwind other exception-handling routines
pthread_atfork
tis and the standard I/O routines
They have certain differences in behavior, as described in Section 9.2.5.1 of the Programmer's Guide.
Routines related to multi-threading or exception-handling (like pthread, exc_* and libmach routines)
Routines that assume a particular environment (like X and Motif)
They may not be useful or correct in an ATOM analysis environment.