|
|
玩足球的风衣 · mysql8.0无法远程连接 The ...· 2 年前 · |
|
|
含蓄的莲藕 · shell 每秒执行一次-掘金· 2 年前 · |
|
|
聪明的猕猴桃 · docker-compose ...· 2 年前 · |
|
|
大鼻子的骆驼 · gitlab或github下fork后如何同 ...· 2 年前 · |
This Technote describes the on-disk format for an HFS Plus volume. It does not describe any programming interfaces for HFS Plus volumes.
This technote is directed at developers who need to work with HFS Plus at a very low level, below the abstraction provided by the File Manager programming interface. This includes developers of disk recovery utilities and programmers implementing HFS Plus support on other platforms.
This technote assumes that you have a conceptual understanding of the HFS volume format, as described in Inside Macintosh: Files .
HFS Plus is a volume format for Mac OS. HFS Plus was introduced with Mac OS 8.1 . HFS Plus is architecturally very similar to HFS, although there have been a number of changes. The following table summarizes the important differences.
Table 1 HFS and HFS Plus Compared
Maintains efficiency in the face of the other changes. (This larger catalog node size is due to the much longer file names [512 bytes as opposed to 32 bytes], and larger catalog records (because of more/larger fields)).
Maximum file size
2 31 bytes
2 63 bytes
Obvious user benefit, especially for multimedia content creators.
The extent to which these HFS Plus features are available through a programming interface is OS dependent. Mac OS versions less than 9.0 do not provide programming interfaces for any HFS Plus-specific features.
To summarize, the key goals that guided the design of the HFS Plus volume format were:
The following sections describes these goals, and the differences between HFS and HFS Plus required to meet these goals.
HFS divides the total space on a volume into equal-sized pieces called allocation blocks. It uses 16-bit fields to identify a particular allocation block, so there must be less than 2 16 (65,536) allocation blocks on an HFS volume. The size of an allocation block is typically the smallest multiple of 512 such that there are less than 65,536 allocation blocks on the volume (i.e., the volume size divided by 65,535, rounded up to a multiple of 512). Any non-empty fork must occupy an integral number of allocation blocks. This means that the amount of space occupied by a fork is rounded up to a multiple of the allocation block size. As volumes (and therefore allocation blocks) get bigger, the amount of allocated but unused space increases.
HFS Plus uses 32-bit values to identify allocation blocks. This allows up to 2 32 (4,294,967,296) allocation blocks on a volume. More allocation blocks means a smaller allocation block size, especially on volumes of 1 GB or larger, which in turn means less average wasted space (the fraction of an allocation block at the end of a fork, where the entire allocation block is not actually used). It also means you can have more files, since the available space can be more finely distributed among a larger number of files. This change is especially beneficial if the volume contains a large number of small files.
HFS uses 31-byte strings to store file names. HFS does not store any kind of script information with the file name to indicate how it should be interpreted. File names are compared and sorted using a routine that assumes a Roman script, wreaking havoc for names that use some other script (such as Japanese). Worse, this algorithm is buggy, even for Roman scripts. The Finder and other applications interpret the file name based on the script system in use at runtime. The problem with using non-Roman scripts in an HFS file name is that HFS compares file names in a case- insensitive fashion. The case-insensitive comparison algorithm assume a MacRoman encoding. When presented with non-Roman text, this algorithm fails in strange ways. The upshot is that HFS decides that certain non-Roman file names are duplicates of other file names, even though they are not duplicates in the source encoding.
HFS Plus uses up to 255 Unicode characters to store file names. Allowing up to 255 characters makes it easier to have very descriptive names. Long names are especially useful when the name is computer-generated (such as Java class names).
The HFS catalog B-tree uses 512-byte nodes. An HFS Plus file name can occupy up to 512 bytes (including the length field). Since a B-tree index node must store at least two keys (plus pointers and node descriptor), the HFS Plus catalog must use a larger node size. The typical node size for an HFS Plus catalog B-tree is 4 KB.
In the HFS catalog B-tree, the keys stored in an index node always occupy a fixed amount of space, the maximum key size. In HFS Plus, the keys in an index node may occupy a variable amount of space determined by the actual size of the key. This allows for less wasted space in index nodes and creates, on typical disks, a substantially larger branching factor in the tree (requiring fewer node accesses to find any given record).
Files on an HFS volume have two forks: a data fork and a resource fork, either of which may be empty (zero length). Files and directories also contain a small amount of additional information (known as catalog information or metadata) such as the modification date or Finder info.
Apple software teams and third-party developers often need to store information associated with particular files and directories. In some cases (for example, custom icons for files), the data or resource fork is appropriate. But in other cases (for example, custom icons for directories, or File Sharing access privileges), using the data or resource fork is not appropriate or not possible.
A number of products have implemented special-purpose solutions for storing their file- and directory-related data. But because these are not managed by the file system, they can become inconsistent with the file and directory structure.
HFS Plus has an attribute file, another B-tree, that can be used to store additional information for a file or directory. Since it is part of the volume format, this information can be kept with the file or directory as is it moved or renamed, and can be deleted when the file or directory is deleted. The contents of the attribute file's records have not been fully defined yet, but the goal is to provide an arbitrary number of forks, identified by Unicode names, for any file or directory.
Note:
Because the attributes file has not been fully
defined yet, current implementations are unable to
delete named forks when a file or directory is
deleted. Future implementations that properly
delete named forks will need to check for these
orphaned named forks and delete them when the
volume is mounted. The
lastMountedVersion
field of the volume
header can be used to detect when such a check
needs to take place.
Whenever possible, an application should delete named forks rather than orphan them.
HFS Plus defines a special startup file , an unstructured fork that can be found easily during system startup. The location and size of the startup file is described in the volume header . The startup file is especially useful on systems that don't have HFS or HFS Plus support in ROM. In many respects, the startup file is a generalization of the HFS boot blocks, one that provides a much larger, variable-sized amount of storage.
HFS Plus uses a number of interrelated structures to manage the organization of data on the volume. These structures include:
Each of these complex structures is described in its own section. The purpose of this section is to give an overview of the volume format, describe how the structures fit together, and define the primitive data types used by HFS Plus.
HFS Plus is a specification of how a volume (files that contain user data, along with the structure to retrieve that data) exists on a disk (the medium on which user data is stored). The storage space on a disk is divided into units called sectors . A sector is the smallest part of a disk that the disk's driver software will read or write in a single operation (without having to read or write additional data before or after the requested data). The size of a sector is usually based on the way the data is physically laid out on the disk. For hard disks, sectors are typically 512 bytes. For optical media, sectors are typically 2048 bytes.
Most of the data structures on an HFS Plus volume do not
depend on the size of a sector, with the exception of the
journal
. Because the journal does rely
on accessing individual sectors, the sector size is stored
in the
jhdr_size
field of the
journal header
(if the
volume has a journal).
HFS Plus allocates space in units called allocation blocks ; an allocation block is simply a group of consecutive bytes. The size (in bytes) of an allocation block is a power of two, greater than or equal to 512, which is set when the volume is initialized. This value cannot be easily changed without reinitializing the volume. Allocation blocks are identified by a 32-bit allocation block number , so there can be at most 2 32 allocation blocks on a volume. Current implementations of the file system are optimized for 4K allocation blocks.
Note:
For the best performance, the allocation block size should
be a multiple of the sector size. If the
volume has an
HFS wrapper
, the
wrapper's allocation block size and allocation block start
should also be multiples of the sector size to
allow the best performance.
All of the volume's structures, including the volume header, are part of one or more allocation blocks (with the possible exception of the alternate volume header, discussed below ). This differs from HFS, which has several structures (including the boot blocks, master directory block, and bitmap) which are not part of any allocation block.
IMPORTANT:
The actual algorithm used to extend files is not part
of this specification. The implementation is not
required to act on the clump values in the volume
header; it merely provides space to store those
values.
Note:
The current non-contiguous algorithm in Mac OS will
begin allocating at the next free block it finds.
It will extend its allocation up to a multiple of
the clump size if there is sufficient free space
contiguous with the end of the requested
allocation. Space is not allocated in contiguous
clump-sized pieces.
Every HFS Plus volume must have a volume header . The volume header contains sundry information about the volume, such as the date and time of the volume's creation and the number of files on the volume, as well as the location of the other key structures on the volume. The volume header is always located at 1024 bytes from the start of the volume.
A copy of the volume header, known as the alternate volume header , is stored starting at 1024 bytes before the end of the volume. The first 1024 bytes of volume (before the volume header), and the last 512 bytes of the volume (after the alternate volume header) are reserved . All of the allocation blocks containing the volume header, alternate volume header, or the reserved areas before the volume header or after the alternate volume header, are marked as used in the allocation file . The actual number of allocation blocks marked this way depends on the allocation block size.
An HFS Plus volume contains five special files , which store the file system structures required to access the file system payload: folders, user files , and attributes. The special files are the catalog file, the extents overflow file, the allocation file, the attributes file and the startup file. Special files only have a single fork (the data fork) and the extents of that fork are described in the volume header.
The catalog file is a special file that describes the folder and file hierarchy on a volume. The catalog file contains vital information about all the files and folders on a volume, as well as the catalog information , for the files and folders that are stored in the catalog file. The catalog file is organized as a B-tree (or "balanced tree") to allow quick and efficient searches through a large folder hierarchy.
The catalog file stores the file and folder names, which consist of up to 255 Unicode characters, as described below .
Note:
The
B-Trees
section contains
an in-depth description of the B-trees used by HFS
Plus.
The attributes file is another special file which contains additional data for a file or folder. Like the catalog file, the attributes file is organized as a B-tree. In the future, it will be used to store information about additional forks. (This is similar to the way the catalog file stores information about the data and resource forks of a file.)
HFS Plus tracks which allocation blocks belong to a fork by maintaining a list of the fork's extents. An extent is a contiguous range of allocation blocks allocated to some fork, represented by a pair of numbers: the first allocation block number and the number of allocation blocks. For a user file, the first eight extents of each fork are stored in the volume's catalog file. Any additional extents are stored in the extents overflow file , which is also organized as a B-tree.
The extents overflow file also stores additional extents for the special files except for the extents overflow file itself. However, if the startup file requires more than the eight extents in the Volume Header (and thus requires additional extents in the extents overflow file), it would be much harder to access, and defeat the purpose of the startup file. So, in practice, a startup file should be allocated such that it doesn't need additional extents in the extents overflow file.
The allocation file is a special file which specifies whether an allocation block is used or free. This performs the same role as the HFS volume bitmap, although making it a file adds flexibility to the volume format.
The startup file is another special file which facilitates booting of non-Mac OS computers from HFS Plus volumes.
Finally, the bad block file prevents the volume from using certain allocation blocks because the portion of the media that stores those blocks is defective. The bad block file is neither a special file nor a user file; this is merely convention used in the extents overflow file. See Bad Block File for more details.
The bulk of an HFS Plus volume consists of seven types of information or areas:
The general structure of an HFS Plus volume is illustrated in Figure 1.
Figure 1 . Organization of an HFS Plus Volumes.
The volume header is always at a fixed location (1024 bytes from the start of the volume). However, the special files can appear anywhere between the volume header block and the alternate volume header block. These files can appear in any order and are not necessarily contiguous.
The information on HFS Plus volumes (with the possible exception of the alternate volume header, as discussed below ) is organized solely in allocation blocks. Allocation blocks are simply a means of grouping space on the media into convenient parcels. The size of an allocation block is a power of two, and at least 512. The allocation block size is a volume header parameter whose value is set when the volume is initialized; it cannot be changed easily without reinitializing the volume.
Note:
The allocation block size is a classic
speed-versus- space tradeoff. Increasing the
allocation block size decreases the size of the
allocation file, and often reduces the number of
separate extents that must be manipulated for every
file. It also tends to increase the average size of
a disk I/O, which decreases overhead. Decreasing
the allocation block size reduces the average
number of wasted bytes per file, making more
efficient use of the volume's space.
This section describes the primitive data types used on an HFS Plus volume. All data structures in this volume are defined in the C language. The specification assumes that the compiler will not insert any padding fields. Any necessary padding fields are explicitly declared.
IMPORTANT:
The HFS Plus volume format is largely derived from
the HFS volume format. When defining the new
format, it was decided to remove unused fields
(primarily legacy MFS fields) and arrange all the
remaining fields so that similar fields were
grouped together and that all fields had proper
alignment (using PowerPC alignment rules).
In many places this specification describes a field, or bit within a field, as reserved. This has a definite meaning, namely:
This definition allows for backward-compatible enhancements to the volume format.
Pad fields have exactly the same semantics as a reserved field. The different name merely reflects the designer's goals when including the field, not the behavior of the implementation.
All integer values are defined by one of the following
primitive types:
UInt8
,
SInt8
,
UInt16
,
SInt16
,
UInt32
,
SInt32
,
UInt64
, and
SInt64
. These
represent unsigned and signed (2's complement) 8-bit,
16-bit, 32-bit, and 64-bit numbers.
All multi-byte integer values are stored in big-endian
format. That is, the bytes are stored in order from most
significant byte through least significant byte, in
consecutive bytes, at increasing offset from the start of a
block. Bits are numbered from 0 to
n
-1 (for types
UInt
n
and
SInt
n
),
with bit 0 being the least significant bit.
File and folder names on HFS Plus consist of up to 255
Unicode characters with a preceding 16-bit length, defined
by the type
HFSUniStr255
.
UniChar
is a
UInt16
that
represents a character as defined in the Unicode character
set defined by
The Unicode Standard, Version 2.0
[Unicode, Inc. ISBN 0-201-48345-9].
HFS Plus stores strings fully decomposed and in canonical order. HFS Plus compares strings in a case-insensitive fashion. Strings may contain Unicode characters that must be ignored by this comparison. For more details on these subtleties, see Unicode Subtleties .
A variant of HFS Plus, called HFSX , allows volumes whose names are compared in a case-sensitive fashion. The names are fully decomposed and in canonical order, but no Unicode characters are ignored during the comparison.
Traditional Mac OS programming interfaces pass filenames as
Pascal strings (either as a
StringPtr
or as a
Str63
embedded in an
FSSpec
). The
characters in those strings are not Unicode; the encoding
varies depending on how the system software was localized
and what language kits are installed. Identical sequences of
bytes can represent vastly different Unicode character
sequences. Similarly, many Unicode characters belong to more
than one Mac OS text encoding.
HFS Plus includes two features specifically designed to
help Mac OS handle the conversion between Mac OS-encoded
Pascal strings and Unicode. The first feature is the
textEncoding
field of the file and folder
catalog records. This field is defined as a hint to be used
when converting the record's Unicode name back to a Mac OS-
encoded Pascal string.
The valid values for the
textEncoding
field
are defined in Table 2.
Table 2 Text Encodings
Non-Mac OS implementations of HFS Plus may choose to simply ignore the
textEncoding
field. In this case, the field must be treated as
a
reserved
field.
Note:
Mac OS uses the
textEncoding
field in
the following way. When a file or folder is created
or renamed, Mac OS converts the supplied Pascal
string to a
HFSUniStr255
. It stores
the source text encoding in the
textEncoding
field of the catalog
record. When Mac OS needs to create a Pascal string
for that record, it uses the
textEncoding
as a hint to the text
conversion process. This hint ensures a high-degree
of round-trip conversion fidelity, which in turn
improves compatibility.
The second use of text encodings in HFS Plus is the
encodingsBitmap
field of the volume header. For
each encoding used by a catalog node on the volume, the
corresponding bit in the
encodingsBitmap
field
must be set.
It is acceptable for a bit in this bitmap to be set even though no names on the volume use that encoding. This means that when an implementation deletes or renames an object, it does not have to clear the encoding bit if that was the last name to use the given encoding.
IMPORTANT:
The text encoding value is used as the number of
the bit to set in
encodingsBitmap
to
indicate that the encoding is used on the volume.
However,
encodingsBitmap
is only 64
bits long, and thus the text encoding values for
MacFarsi and MacUkrainian cannot be used as bit
numbers. Instead, another bit number (shown in
parenthesis) is used.
Note:
Mac OS uses the
encodingsBitmap
field
to determine which text encoding conversion tables
to load when the volume is mounted. Text encoding
conversion tables are large, and loading them
unnecessarily is a waste of memory. Most systems
only use one text encoding, so there is a
substantial benefit to recording which encodings
are required on a volume-by-volume basis.
encodingsBitmap
field. Specifically, if the implementation sets the
textEncoding
field a catalog record to
a text-encoding value, it must ensure that the
corresponding bit is set in
encodingsBitmap
to ensure correct
operation when that disk is mounted on a system
running Mac OS.
HFS Plus stores dates in several data structures,
including the volume header and catalog records. These dates
are stored in unsigned 32-bit integers (
UInt32
)
containing the number of seconds since midnight, January 1,
1904, GMT. This is slightly different from HFS, where the
value represents local time.
The maximum representable date is February 6, 2040 at 06:28:15 GMT.
The date values do not account for leap seconds. They do include a leap day in every year that is evenly divisible by four. This is sufficient given that the range of representable dates does not contain 1900 or 2100, neither of which have leap days.
The implementation is responsible for converting these times to the format expected by client software. For example, the Mac OS File Manager passes dates in local time; the Mac OS HFS Plus implementation converts dates between local time and GMT as appropriate.
Note:
The creation date stored in
the Volume Header is NOT stored in GMT; it is
stored in local time. The reason for this is that
many applications (including backup utilities) use
the volume's creation date as a relatively unique
identifier. If the date was stored in GMT, and
automatically converted to local time by an
implementation (like Mac OS), the value would
appear to change when the local time zone or
daylight savings time settings change (and thus
cause some applications to improperly identify the
volume). The use of the volume's creation date as a
unique identifier outweighs its use as a date. This
change was introduced late in the Mac OS 8.1
project.
For each file and folder, HFS Plus maintains a record
containing access permissions, defined by the
HFSPlusBSDInfo
structure.
groupID
adminFlags
st_flags
field of
struct stat
in Mac OS X. See the
manual page for
chflags(2)
for more information. The following table
gives the bit position in the
adminFlags
field and the name of the
corresponding mask used in the
st_flags
field.
Bit
st_flags
mask Meaning
0 SF_ARCHIVED File has been archived
1 SF_IMMUTABLE File may not be changed
2 SF_APPEND Writes to file may only append
ownerFlags
st_flags
field of
struct stat
in Mac OS X. See the
manual page for
chflags(2)
for more information. The following table
gives the bit position in the
ownerFlags
field and the name of the
corresponding mask used in the
st_flags
field.
Bit
st_flags
mask Meaning
0 UF_NODUMP Do not dump (back up or archive) this file
1 UF_IMMUTABLE File may not be changed
2 UF_APPEND Writes to file may only append
3 UF_OPAQUE Directory is opaque (see
below
)
fileMode
#define S_ISUID 0004000 /* set user id on execution */
#define S_ISGID 0002000 /* set group id on execution */
#define S_ISTXT 0001000 /* sticky bit */
#define S_IRWXU 0000700 /* RWX mask for owner */
#define S_IRUSR 0000400 /* R for owner */
#define S_IWUSR 0000200 /* W for owner */
#define S_IXUSR 0000100 /* X for owner */
#define S_IRWXG 0000070 /* RWX mask for group */
#define S_IRGRP 0000040 /* R for group */
#define S_IWGRP 0000020 /* W for group */
#define S_IXGRP 0000010 /* X for group */
#define S_IRWXO 0000007 /* RWX mask for other */
#define S_IROTH 0000004 /* R for other */
#define S_IWOTH 0000002 /* W for other */
#define S_IXOTH 0000001 /* X for other */
#define S_IFMT 0170000 /* type of file mask */
#define S_IFIFO 0010000 /* named pipe (fifo) */
#define S_IFCHR 0020000 /* character special */
#define S_IFDIR 0040000 /* directory */
#define S_IFBLK 0060000 /* block special */
#define S_IFREG 0100000 /* regular */
#define S_IFLNK 0120000 /* symbolic link */
#define S_IFSOCK 0140000 /* socket */
#define S_IFWHT 0160000 /* whiteout */
In some versions of Unix, the sticky bit, S_ISTXT, is used
to indicate that an executable file's code should remain in memory
after the executable finishes; this can help performance if the same
executable is used again soon. Mac OS X does not use this optimization.
If the sticky bit is set for a directory, then Mac OS X restricts
movement, deletion, and renaming of files in that directory.
Files may be removed or renamed only if the user has write access
to the directory; and is the owner of the file or the directory,
or is the super-user.
specialiNodeNumlinkCountrawDeviceS_IFMT
field contains S_IFCHR or S_IFBLK), this field
contains the device number.S_IFWHT and UF_OPAQUE
values are used when the file system is mounted as part
of a union mount. A union mount presents the
combination (union) of several file systems as a single
file system. Conceptually, these file systems are
layered, one on top of another. If a file or directory
appears in multiple layers, the one in the top most
layer is used. All changes are made to the top most
file system only; the others are read-only. To delete a
file or directory that appears in a layer other than the
top layer, a whiteout entry (file type
S_IFWHT) is created in the top layer. If a
directory that appears in a layer other than the top
layer is deleted and later recreated, the contents in
the lower layer must be hidden by setting the
UF_OPAQUE flag in the directory in the top
layer. Both S_IFWHT and
UF_OPAQUE hide corresponding names in lower
layers by preventing a union mount from accessing the
same file or directory name in a lower layer.
Note:
If the S_IFMT field (upper 4 bits) of the fileMode
field is zero, then Mac OS X assumes that the permissions structure is
uninitialized, and internally uses default values for all of the fields.
The default user and group IDs are 99, but can be changed at the time the
volume is mounted. This default ownerID is then subject to
substitution as described above.
This means that files created by Mac OS 8 and 9, or any other implementation that sets the permissions fields to zeroes, will behave as if the "ignore ownership" option is enabled for those files, even if "ignore ownership" is disabled for the volume as a whole.
HFS Plus maintains information about the contents of a
file using the HFSPlusForkData structure. Two
such structures -- one for the resource and one for the data
fork -- are stored in the catalog record for each user file.
In addition, the volume header contains a fork data
structure for each special file.
An unused extent descriptor in an extent record would
have both startBlock and
blockCount set to zero. For example, if a given
fork occupied three extents, then the last five extent
descriptors would be all zeroes.
struct HFSPlusForkData {
UInt64 logicalSize;
UInt32 clumpSize;
UInt32 totalBlocks;
HFSPlusExtentRecord extents;
typedef struct HFSPlusForkData HFSPlusForkData;
typedef HFSPlusExtentDescriptor HFSPlusExtentRecord[8];
HFSPlusForkData structures in the
volume header, this is the fork's
clump size, which is used in preference to the
default clump size in the volume header.HFSPlusForkData structures in a
catalog record, this field was intended to store a per-fork
clump size to override the default clump size
in the volume header. However, Apple
implementations prior to Mac OS X version 10.3 ignored this field.
As of Mac OS X version 10.3, this field is used to keep track of the
number of blocks actually read from the fork. See the Hot
Files section for more information.
totalBlocksextentsstruct HFSPlusExtentDescriptor {
UInt32 startBlock;
UInt32 blockCount;
typedef struct HFSPlusExtentDescriptor HFSPlusExtentDescriptor;
Each HFS Plus volume contains a volume header 1024 bytes from the start of the volume. The volume header -- analogous to the master directory block (MDB) for HFS -- contains information about the volume as a whole, including the location of other key structures in the volume. The implementation is responsible for ensuring that this structure is updated before the volume is unmounted.
A copy of the volume header, the alternate volume header, is stored starting 1024 bytes before the end of the volume. The implementation should only update this copy when the length or location of one of the special files changes. The alternate volume header is intended for use solely by disk repair utilities.
The first 1024 bytes and the last 512 bytes of the volume are reserved.
Note:
The first 1024 bytes are reserved for use as boot
blocks; the traditional Mac OS Finder will write to them when
the System Folder changes. The boot block format is
outside the scope of this specification. It is
defined in
Inside
Macintosh: Files.
The last 512 bytes were used during Apple's CPU manufacturing process.
The allocation block (or blocks) containing the first 1536 bytes (reserved space plus volume header) are marked as used in the allocation file (see the Allocation File section). Also, in order to accommodate the alternate volume header and the reserved space following it, the last allocation block (or two allocation blocks, if the volume is formatted with 512-byte allocation blocks) is also marked as used in the allocation file.
The alternate volume header is always stored at offset 1024 bytes from the end of the volume. If the disk size is not an even multiple of the allocation block size, this area may lie beyond the last allocation block. However, the last allocation block (or two allocation blocks for a volume formatted with 512-byte allocation blocks) is still reserved even if the alternate volume header is not stored there.The volume header is described by the
HFSPlusVolumeHeader type.
kHFSPlusSigWord
(
'H+'
) for an
HFS Plus volume, or
kHFSXSigWord
(
'HX'
)
for an
HFSX
volume.
version
kHFSPlusVersion
) for HFS Plus volumes, or
5 (
kHFSXVersion
) for
HFSX
volumes.
attributes
lastMountedVersion
'8.10'
.
The value used by Mac OS X is
'10.0'
. The
value used by a
journaled
volume
(including
HFSX
) in Mac OS X is
'HFSJ'
.
The value used by fsck_hfs on Mac OS X is
'fsck'
.
Note:
It is very important for implementations (and
utilities that directly modify the volume!) to set
the
lastMountedVersion
. It is also
important to choose different values when
non-trivial changes are made to an implementation
or utility. If a bug is found in an implementation
or utility, and it sets the
lastMountedVersion
correctly, it will
be much easier for other implementations and
utilities to detect and correct any problems.
journalInfoBlock
JournalInfoBlock
for this volume's journal. This field is valid only if bit
kHFSVolumeJournaledBit
is set in the
attribute
field; otherwise, this field is
reserved
.
createDate
modifyDate
backupDate
checkedDate
fileCount
fileCount
field does not include the special
files. It should equal the number of file records found
in the catalog file.
folderCount
folderCount
field does not include the
root folder. It should equal the number of folder records
in the catalog file, minus one (since the root folder has
a folder record in the catalog file).
blockSize
totalBlocks
freeBlocks
nextAllocation
nextAllocation
field is used by Mac OS as a
hint for where to start searching for free allocation blocks
when allocating space for a file. It contains the allocation
block number where the search should begin. An
implementation that doesn't want to use this kind of hint
can just treat the field as
reserved
. [Implementation
details: traditional Mac OS implementations typically
set it to the first allocation block of the extent most
recently allocated. It is not set to the allocation block
immediately following the most recently allocated extent
because of the likelihood of that extent being shortened
when the file is closed (since a whole
clump
may have been allocated but not
actually used).] See
Allocation
File
section for details.
rsrcClumpSize
rsrcClumpSize
and use
dataClumpSize
for both data and resource
forks.
dataClumpSize
rsrcClumpSize
and use
dataClumpSize
for both data and resource
forks.
nextCatalogID
writeCount
writeCount
field if it modifies the volume's
structures directly. This is particularly important if it
adds or deletes items on the volume.
encodingsBitmap
finderInfo
finderInfo[0]
contains the directory ID of the
directory containing the bootable system (for example, the
System Folder in Mac OS 8 or 9, or
/System/Library/CoreServices
in Mac OS X). It is zero if there is no bootable system on the volume.
This value is typically equal to either
finderInfo[3]
or
finderInfo[5]
.
finderInfo[1]
contains the parent directory ID of
the startup application (for example, Finder), or zero if the volume
is not bootable.
finderInfo[2]
contains the directory ID of a directory
whose window should be displayed in the Finder when the volume is
mounted, or zero if no directory window should be opened. In
traditional Mac OS, this is the first in a linked list of windows
to open; the
frOpenChain
field of the directory's
Finder Info
contains the next directory ID
in the list. The open window list is deprecated. The Mac OS X
Finder will open this directory's window, but ignores the rest
of the open window list. The Mac OS X Finder does not modify
this field.
finderInfo[3]
contains the directory ID of a bootable
Mac OS 8 or 9 System Folder, or zero if there isn't one.
finderInfo[4]
is
reserved
.
finderInfo[5]
contains the directory ID of a bootable
Mac OS X system (the
/System/Library/CoreServices
directory), or zero if there is no bootable Mac OS X system on
the volume.
finderInfo[6]
and
finderInfo[7]
are
used by Mac OS X to contain a 64-bit unique volume identifier.
One use of this identifier is for tracking whether a given
volume's ownership (user ID) information should be honored.
These elements may be zero if no such identifier has been
created for the volume.
allocationFile
HFSPlusForkData
type.
extentsFile
HFSPlusForkData
type.
catalogFile
HFSPlusForkData
type.
attributesFile
HFSPlusForkData
type.
startupFile
HFSPlusForkData
type.
The
attributes
field of a volume header is
treated as a set of one-bit flags. The definition of the
bits is given by the constants listed below.
kHFSVolumeSparedBlocksBit
(bit 9)
kHFSBadBlockFileID
). See
Bad Block File
for details.
kHFSVolumeNoCacheRequiredBit
(bit 10)
kHFSBootVolumeInconsistentBit
(bit 11)
kHFSVolumeUnmountedBit
, but inverted in
meaning. An implementation must set this bit on the media
when it mounts a volume for writing. An implementation
must clear this bit on the media as the last step of
unmounting a writable volume, after all other volume
information has been flushed. If an implementation is
asked to mount a volume where this bit is set, it must
assume the volume is inconsistent, and do appropriate
consistency
checking
before using the volume.
kHFSCatalogNodeIDsReusedBit
(bit 12)
nextCatalogID
field overflows 32 bits, forcing smaller
catalog node IDs
to be reused. When this
bit is set, it is common (and not an error) for catalog
records to exist with IDs greater than or equal to
nextCatalogID
. If this bit is set, you must
ensure that IDs assigned to newly created catalog records do
not conflict with IDs used by existing records.
kHFSVolumeJournaledBit
(bit 13)
journalInfoBlock
field of the Volume Header.
kHFSVolumeSoftwareLockBit
(bit 15)
Note:
Mac OS X versions 10.0 to 10.3 don't properly honor
kHFSVolumeSoftwareLockBit
. They incorrectly
allow such volumes to be modified. This bug is expected
to be fixed in a future version of Mac OS X. (r. 3507614)
Note:
An implementation may keep a copy of the attributes
in memory and use bits 0-7 for its own runtime
flags. As an example, Mac OS uses bit 7,
kHFSVolumeHardwareLockBit
, to indicate
that the volume is write-protected due to some
hardware setting.
kHFSVolumeUnmountedBit
and
kHFSBootVolumeInconsistentBit
)
deserves an explanation. Macintosh ROMs check the
consistency of a boot volume if
kHFSVolumeUnmountedBit
is clear. The
ROM-based check is very slow, annoyingly so. This
checking code was significantly optimized in Mac OS
7.6. To prevent the ROM check from being used, Mac
OS 7.6 (and higher) leaves the original consistency
check bit (
kHFSVolumeUnmountedBit
) set
at all times. Instead, an alternative flag
(
kHFSBootVolumeInconsistentBit
) is
used to signal that the disk needs a consistency
check.
kHFSBootVolumeInconsistentBit
should
be used as described but
kHFSVolumeUnmountedBit
should remain
set; for all other volumes, use the
kHFSVolumeUnmountedBit
as described
but keep the
kHFSBootVolumeInconsistentBit
clear.
This is an optimization that prevents the Mac OS
ROM from doing a very slow consistency check when
the boot volume is mounted since it only checks
kHFSVolumeUnmountedBit
, and won't do a
consistency check; later on, the File Manager will
see the
kHFSBootVolumeInconsistentBit
set and do a better, faster consistency check. (It
would be OK to always use both bits at the expense
of a slower Mac OS boot.)
Note:
For a practical description of the algorithms used
to maintain a B-tree, see
Algorithms in
C
, Robert Sedgewick, Addison-Wesley, 1992.
ISBN: 0201514257.
Many textbooks describe B-trees in which an index node contains N keys and N+1 pointers, and where keys less than key #X lie in the subtree pointed to by pointer #X, and keys greater than key #X lie in the subtree pointed to by pointer #X+1. (The B-tree implementor defines whether to use pointer #X or #X+1 for equal keys.)
HFS and HFS Plus are slightly different; in a given subtree, there are no keys less than the first key of that subtree's root node.
This section describes the B-tree structure used for the
catalog, extents overflow, and attributes files. A B-tree is
stored in file data fork. Each B-tree has a
HFSPlusForkData
structure in the volume header that describes the size and
initial extents of that data fork.
Note:
Special files do not have a resource fork because
there is no place to store its
HFSPlusForkData
in the volume header.
However, it's still important that the B-tree is in
the data fork because the fork is part of the key
used to store B-tree extents in the extents
overflow file.
A B-tree file is divided up into fixed-size nodes , each of which contains records , which consist of a key and some data. The purpose of the B-tree is to efficiently map a key into its corresponding data. To achieve this, keys must be ordered, that is, there must be a well-defined way to decide whether one key is smaller than, equal to, or larger than another key.
The node size (which is expressed in bytes) must be power of two, from 512 through 32,768, inclusive. The node size of a B-tree is determined when the B-tree is created. The logical length of a B-tree file is just the number of nodes times the node size.
There are four kinds of nodes.
All nodes share a common structure, described in the next section.
Nodes are indicated by number. The node's number can be calculated by dividing its offset into the file by the node size. Each node has the same general structure, consisting of three main parts: a node descriptor at the beginning of the node, a list of record offsets at the end of the node, and a list of records. This structure is depicted in Figure
The
node descriptor
contains basic information
about the node as well as forward and backward links to
other nodes. The
BTNodeDescriptor
data type
describes this structure.
numRecords
reserved
A node descriptor is always 14 (which is
sizeof(BTNodeDescriptor)
) bytes long, so the
list of records
contained in a node always starts 14
bytes from the start of the node. The size of each record
can vary, depending on the record's type and the amount of
information it contains.
The records are accessed using the
list of record
offsets
at the end of the node. Each entry in this list
is a
UInt16
which contains the offset, in
bytes, from the start of the node to the start of the
record. The offsets are stored in reverse order, with the
offset for the first record in the last two bytes of the
node, the offset for the second record is in the previous
two bytes, and so on. Since the first record is always at
offset 14, the last two bytes of the node contain the value
IMPORTANT:
The list of record offsets always contains one more
entry than there is records in the node. This entry
contains the offset to the first byte of free space
in the node, and thus indicates the size of the
last record in the node. If there is no free space
in the node, the entry contains its own byte offset
from the start of the node.
The
kind
field of the node descriptor
describes the type of a node, which indicates what kinds of
records it contains and, therefore, its purpose in the
B-tree hierarchy. There are four kinds of node types given
by the following constants:
It's important to realise that the B-tree node type determines the type of records found in the node. Leaf nodes always contain data records. Index nodes always contain pointer records. Map nodes always contain map records. The header node always contains a header record, a reserved record, and a map record. The four node types and their corresponding records are described in the subsequent sections.
The first node (node 0) in every B-tree file is a header node, which contains essential information about the entire B-tree file. There are three records in the header node. The first record is the B-tree header record. The second record is the user data record and is always 128 bytes long. The last record is the B-tree map record; it occupies all of the remaining space between the user data record and the record offsets. The header node is shown in Figure 3.
Figure 3 Header node structure
The
fLink
field of the header node's node
descriptor contains the node number of the first map node,
or 0 if there are no map nodes. The
bLink
field
of the header node's node descriptor must be set to zero.
The B-tree
header record
contains general
information about the B-tree such as its size, maximum key
length, and the location of the first and last leaf nodes.
The data type
BTHeaderRec
describes the
structure of a header record.
Note:
The root node can be a leaf node (in the case where
there is only a single leaf node, and therefore no
index nodes, as might happen with the catalog file
on a newly initialized volume). If a tree has no
leaf nodes (like the extents overflow file on a
newly initialized volume), the
firstLeafNode
,
lastLeafNode
, and
rootNode
fields will all be zero. If
there is only one leaf node (as may be the case
with the catalog file on a newly initialized
volume),
firstLeafNode
,
lastLeafNode
, and
rootNode
will all have the same value
(i.e., the node number of the sole leaf node). The
firstLeafNode
and
lastLeafNode
fields just make it easy
to walk through all the leaf nodes by just
following
fLink/bLink
fields.
The fields have the following meaning:
treeDepth
height
field of the root node.
rootNode
rootNode
is a leaf
node. See
Inside
Macintosh: Files
, pp. 2-69 for details.
leafRecords
firstLeafNode
lastLeafNode
nodeSize
maxKeyLength
maxKeyLength
values for
the catalog and extents files for both HFS and HFS Plus
(
kHFSPlusExtentKeyMaximumLength
,
kHFSExtentKeyMaximumLength
,
kHFSPlusCatalogKeyMaximumLength
,
kHFSCatalogKeyMaximumLength
). The maximum
key length for the attributes B-tree will probably be a
little larger than for the catalog file. In general,
maxKeyLength
has to be small enough
(compared to
nodeSize
) so that a single node
can fit two keys of maximum size plus the node descriptor
and offsets.
totalNodes
nodeSize
.
freeNodes
reserved1
clumpSize
clumpSize
field of the
HFSPlusForkData
record is used instead. For maximum compatibility, an
implementation should probably set the
clumpSize
in the node descriptor to the same
value as the
clumpSize
in the
HFSPlusForkData
when initializing a volume.
Otherwise, it should treat the header records's
clumpSize
as reserved.
btreeType
BTreeTypes
:
enum BTreeTypes{
kHFSBTreeType = 0, // control file
kUserBTreeType = 128, // user btree type starts from 128
kReservedBTreeType = 255
This field must be equal to kHFSBTreeType
for the catalog, extents, and attributes B-trees. This field
must be equal to kUserBTreeType for the hot file B-tree. Historically, values of
1 to 127 and kReservedBTreeType were used in
B-trees used by system software in Mac OS 9 and earlier.
keyCompareTypekHFSCaseFolding 0xCF Case folding (case-insensitive)
kHFSBinaryCompare 0xBC Binary compare (case-sensitive)
attributesreserved3The following constants define the various bits that may
be set in the attributes field of the header
record.
kBTBigKeysMaskkeyLength field
of the keys in index and leaf nodes is
UInt16; otherwise, it is a
UInt8. This bit must be set for all HFS Plus
B-trees.kBTVariableIndexKeysMaskkeyLength field; otherwise, the keys in
index nodes always occupy maxKeyLength
bytes. This bit must be set for the HFS Plus Catalog
B-tree, and cleared for the HFS Plus Extents B-tree.Bits not specified here must be treated as reserved.
The second record in a header node is always 128 bytes long. It provides a small space to store information associated with a B-tree.
In the HFS Plus catalog, extents, and attributes B-trees, this record is unused and reserved. In the HFS Plus hot file B-tree, this record contains general information about the hot file recording process.
The remaining space in the header node is occupied by a third record, the map record. It is a bitmap that indicates which nodes in the B-tree are used and which are free. The bits are interpreted in the same way as the bits in the allocation file.
All tolled, the node descriptor, header record, reserved
record, and record offsets occupy 256 bytes of the header
node. So the size of the map record (in bytes) is
nodeSize minus 256. If there are more nodes in
the B-tree than can be represented by the map record in the
header node, map nodes are used to store additional
allocation data.
If the map record of the header node is not large enough
to represent all of the nodes in the B-tree, map
nodes are used to store the remaining allocation data.
In this case, the fLink field of the header
node's node descriptor contains the node number of the first
map node.
A map node consists of the node descriptor and a single map record. The map record is a continuation of the map record contained in the header node. The size of the map record is the size of the node, minus the size of the node descriptor (14 bytes), minus the size of two offsets (4 bytes), minus two bytes of free space. That is, the size of the map record is the size of the node minus 20 bytes; this keeps the length of the map record an even multiple of 4 bytes. Note that the start of the map record is not aligned to a 4-byte boundary: it starts immediately after the node descriptor (at an offset of 14 bytes).
The B-tree uses as many map nodes as needed to provide
allocation data for all of the nodes in the B-tree. The map
nodes are chained through the fLink fields of
their node descriptors, starting with the header node. The
fLink field of the last map node's node
descriptor is zero. The bLink field is not used
for map nodes and must be set to zero for all map nodes.
Not using the bLink field is
consistent with the HFS volume format, but not
really consistent with the overall design.
The records in index and leaf nodes share a common
structure. They contain a keyLength, followed
by the key itself, followed by the record data.
The first part of the record, keyLength, is
either a UInt8 or a UInt16,
depending on the attributes field in the
B-tree's header record. If the kBTBigKeysMask
bit is set in attributes, the
keyLength is a UInt16; otherwise,
it's a UInt8. The length of the key, as stored
in this field, does not include the size of the
keyLength field itself.
IMPORTANT:
All HFS Plus B-trees use a UInt16 for
their key length.
Immediately following the keyLength is the
key itself. The length of the key is determined by the node
type and the B-tree attributes. In leaf nodes, the length is
always determined by keyLength. In index nodes,
the length depends on the value of the
kBTVariableIndexKeysMask bit in the B-tree
attributes in the header record.
If the bit is clear, the key occupies a constant number of
bytes, determined by the maxKeyLength field of
the B-tree header record. If the bit is set, the key length
is determined by the keyLength field of the
keyed record.
Following the key is the record's data. The format of
this data depends on the node type, as explained in the next
two sections. However, the data is always aligned on a
two-byte boundary and occupies an even number of bytes. To
meet the first alignment requirement, a pad byte must be
inserted between the key and the data if the size of the
keyLength field plus the size of the key is
odd. To meet the second alignment requirement, a pad byte
must be added after the data if the data size is odd.
The records in an index node are called pointer
records. They contain a keyLength, a key,
and a node number, expressed a UInt32. The node
whose number is in a pointer record is called a child
node of the index node. An index node has two or more
children, depending on the size of the node and the size of
the keys in the node.
Note:
A root node does not need to exist (if the tree is
empty). And even if one does exist, it need not
be an index node (i.e., it could be a leaf node
if all the records fit in a single node).
The bottom level of a B-tree is occupied exclusively by
leaf nodes, which contain data records instead
of pointer records. The data records contain a
keyLength, a key, and the data associated with
that key. The data may be of variable length.
In an HFS Plus B-tree, the data in the data record is the
HFS Plus volume structure (such as a
CatalogRecord, ExtentRecord, or
AttributeRecord) associated with the key.
A B-tree is highly structured to allow for efficient searching, insertion, and removal. This structure primarily affects the keyed records (pointer records and data records) and the nodes in which they are stored (index nodes and leaf nodes). The following are the ordering requirements for index and leaf nodes:
height field is the same) must be chained
via their fLink and bLink
field. The node with the smallest keys must be first in
the chain and its bLink field must be zero.
The node with the largest keys must be last in the chain
and its fLink field must be zero.fLink). Similarly, all the
keys in the node must be greater than all the keys in the
previous node in the chain (pointed to by
bLink).Keeping the keys ordered in this way makes it possible to quickly search the B-tree to find the data associated with a given key. Figure 4 shows a sample B-tree containing hypothetical keys (in this case, the keys are simply integers).
When an implementation needs to find the data associated with a particular search key, it begins searching at the root node. Starting with the first record, it searches for the record with the greatest key that is less than or equal to the search key. In then moves to the child node (typically an index node) and repeats the same process.
This process continues until a leaf node is reached. If the key found in the leaf node is equal to the search key, the found record contains the desired data associated with the search key. If the found key is not equal to the search key, the search key is not present in the B-tree.

Figure 4. A sample B-Tree
The structure of the B-trees on an HFS Plus volume is a closely related to the B-tree structure used on an HFS volume. There are three principal differences: the size of nodes, the size of keys within index nodes, and the size of a key length (UInt8 vs. UInt16).
In an HFS B-tree, nodes always have a fixed size of 512 bytes.
In an HFS Plus B-tree, the node size is determined by a
field (nodeSize) in the header node. The node
size must be a power from 512 through 32,768. An
implementation must use the nodeSize field to
determine the actual node size.
Note:
The header node is always located at the start of
the B-tree, so you can find it without knowing the
B-tree node size.
HFS Plus uses the following default node sizes:
These sizes are set when the volume is initialized and cannot be easily changed. It is legal to initialize an HFS Plus volume with different node sizes, but the node sizes must be large enough for an index node to contain two keys of maximum size (plus the other overhead such as a node descriptor, record offsets, and pointers to children).
IMPORTANT:
The node size of the catalog file must be at least
kHFSPlusCatalogMinNodeSize (4096).
In an HFS B-tree, all of the keys in an index node occupy a fixed amount of space: the maximum key length for that B-tree. This simplifies the algorithms for inserting and deleting records because, within an index node, one key can be replaced by another key without worrying whether there is adequate room for the new key. However, it is also somewhat wasteful when the keys are variable length (such as in the catalog file, where the key length varies with the length of the file name).
In an HFS Plus B-tree, the keys in an index node are allowed to vary in size. This complicates the algorithms for inserting and deleting records, but reduces wasted space when the length of a key can vary (such as in the catalog file). It also means that the number of keys in an index node will vary with the actual size of the keys.
HFS Plus uses a catalog file to maintain information about the hierarchy of files and folders on a volume. A catalog file is organized as a B-tree file, and hence consists of a header node, index nodes, leaf nodes, and (if necessary) map nodes. The location of the first extent of the catalog file (and hence of the file's header node) is stored in the volume header. From the catalog file's header node, an implementation can obtain the node number of the root node of the B-tree. From the root node, an implementation can search the B-tree for keys, as described in the previous section.
The B-Trees chapter defined a standard rule for the
node size of HFS Plus B-trees. As
the catalog file is a B-tree, it inherits the requirements
of this rule. In addition, the node size of the catalog file
must be at least 4 KB
(kHFSPlusCatalogMinNodeSize).
The catalog node ID is defined by the
CatalogNodeID data type.
The first 16 CNIDs are reserved for use by Apple Computer, Inc., and include the following standard assignments:
kHFSRootParentID = 1, kHFSRootFolderID = 2, kHFSExtentsFileID = 3, kHFSCatalogFileID = 4, kHFSBadBlockFileID = 5, kHFSAllocationFileID = 6, kHFSStartupFileID = 7, kHFSAttributesFileID = 8, kHFSRepairCatalogFileID = 14, kHFSBogusExtentFileID = 15, kHFSFirstUserCatalogNodeID = 16kHFSAllocationFileIDkHFSStartupFileIDkHFSAttributesFileIDkHFSRepairCatalogFileIDfsck_hfs when
rebuilding the catalog file.kHFSBogusExtentFileIDExchangeFiles
operations.kHFSFirstUserCatalogNodeIDIn addition, the CNID of zero is never used and serves as a nil value.
Typically, CNIDs are allocated sequentially, starting at
kHFSFirstUserCatalogNodeID. Versions of the HFS
Plus specification prior to Jan. 18, 2000, required the
nextCatalogID field of the volume header to be greater than the
largest CNID used on the volume (so that an implementation
could use nextCatalogID to determine the CNID to
assign to a newly created file or directory). However, this
can be a problem for volumes that create files or directories
at a high rate (for example, a busy server), since they might
run out of CNID values.
HFS Plus volumes now allow CNID values to wrap around and be
reused. The kHFSCatalogNodeIDsReusedBit in the
attributes field of the
volume header is set to indicate when CNID values have
wrapped around and been reused. When
kHFSCatalogNodeIDsReusedBit is set, the
nextCatalogID field is no longer required to be
greater than any existing CNID.
When kHFSCatalogNodeIDsReusedBit is set,
nextCatalogID may still be used as a hint for the
CNID to assign to newly created files or directories, but the
implementation must verify that CNID is not currently in use
(and pick another value if it is in use). When CNID number
nextCatalogID is already in use, an implementation
could just increment nextCatalogID until it finds
a CNID that is not in use. If nextCatalogID
overflows to zero, kHFSCatalogNodeIDsReusedBit
must be set and nextCatalogID set to
kHFSFirstUserCatalogNodeID (to avoid using any
reserved CNID values).
Note:
Mac OS X versions 10.2 and later, and all versions
of Mac OS 9 support
kHFSCatalogNodeIDsReusedBit.
As the catalog file is a B-tree file, it inherits its basic structure from the definition in B-Trees. Beyond that, you need to know only two things about an HFS Plus catalog file to interpret its data:
For a given file, folder, or thread record, the catalog file
key consists of the parent folder's CNID
and the name of the file or folder. This structure is described
using the HFSPlusCatalogKey type.
keyLength field is required by all
keyed records in a B-tree.
The catalog file, in common with all HFS Plus B-trees,
uses a large key length (UInt16).parentIDnodeNameparentID
folder. For thread records, this is the empty string.nodeName field;
it occupies only the number of bytes required to
hold the name. The keyLength field
determines the actual length of the key; it varies
between
kHFSPlusCatalogKeyMinimumLength (6) to
kHFSPlusCatalogKeyMaximumLength (516).
Note:
The catalog file key mirrors the standard way you
specify a file or folder with the Mac OS File
Manager programming interface, with the exception
of the volume reference number, which determines
which volume's catalog to search.
Catalog file keys are compared first by
parentID and then by nodeName. The
parentID is compared as an unsigned 32-bit
integer. For case-sensitive HFSX volumes,
the characters in nodeName are compared as a
sequence of unsigned 16-bit integers. For case-insensitive
HFSX volumes and HFS Plus volumes, the nodeName
must be compared in a case-insensitive way, as described in the
Case-Insensitive String
Comparison Algorithm section.
For more information about how catalog keys are used to find file, folder, and thread records within the catalog tree, see Catalog Tree Usage.
A catalog file leaf node can contain four different types of data records:
Each record starts with a recordType field,
which describes the type of catalog data record. The
recordType field contains one of the following
values:
Note:
The position of the recordType field,
and the constants chosen for the various record
types, are especially useful if you're writing
common code to handle HFS and HFS Plus volumes.
In HFS, the record type field is one byte, but it's always followed by a one-byte reserved field whose value is always zero. In HFS Plus, the record type field is two bytes. You can use the HFS Plus two-byte record type to examine an HFS record if you use the appropriate constants, as shown below. kHFSFolderRecord = 0x0100, kHFSFileRecord = 0x0200, kHFSFolderThreadRecord = 0x0300, kHFSFileThreadRecord = 0x0400
HFSCatalogFolder type to
interpret the data.kHFSFileRecordHFSCatalogFile type to
interpret the data.kHFSFolderThreadRecordHFSCatalogThread
type to interpret the data.kHFSFileThreadRecordHFSCatalogThread
type to interpret the data.The catalog folder record is used in the catalog B-tree
file to hold information about a particular folder on the
volume. The data of the record is described by the
HFSPlusCatalogFolder type.
valenceparentID is equal
to this folder's folderID.PBGetCatInfo. As a practical
matter, many programs are likely to fails with
anywhere near that many items in a single folder.
So, the volume format allows more than 32,767 items
in a folder, but it's probably not a good idea to
exceed that limit right now.folderIDcreateDatecreateDate of the Volume Header is NOT
stored in GMT; it is local time. (Further, if the volume
has an HFS wrapper, the creation date in the MDB should
be the same as the createDate in the Volume
Header).contentModDatecontentModDate when getting and
setting the modification date. The traditional Mac OS
APIs treat attributeModDate as a
reserved field.
attributeModDatest_ctime field of struct stat).
All versions of Mac OS 8 and 9 treat this field as reserved. See
HFS Plus Dates for a description of
the format.accessDateaccessDate field. Folders
created by traditional Mac OS have an
accessDate of zero.
backupDatepermissionspermissions field. Folders created
by traditional Mac OS have the entire field set to 0.
userInfofinderInfotextEncodingreservedThe catalog file record is used in the catalog B-tree
file to hold information about a particular file on the
volume. The data of the record is described by the
HFSPlusCatalogFile type.
reserved1fileIDcreateDatecontentModDateattributeModDatest_ctime field of struct stat).
All versions of Mac OS 8 and 9 treat this field as reserved. See
HFS Plus Dates for a description of
the format.accessDateaccessDate field. Files
created by traditional Mac OS have an
accessDate of zero.backupDatepermissionspermissions field. Files created
by traditional Mac OS have the entire field set to 0.
userInfofinderInfotextEncodingreserved2dataForkHFSPlusForkData type.resourceForkHFSPlusForkData type.For each fork, the first eight extents are described by
the HFSPlusForkData field in the catalog file
record. If a fork is sufficiently fragmented to require more
than eight extents, the remaining extents are described by
extent records in the extents
overflow file.
kHFSFileLockedBit is set, then none
of the forks may be extended, truncated, or written to.
They may only be opened for reading (not for writing).
The catalog information (like finderInfo and
userInfo) may still be changed.kHFSThreadExistsBit,
kHFSThreadExistsMaskThe catalog thread record is used in the catalog B-tree file
to link a CNID to the file or folder record
using that CNID. The data of the record is described by the
HFSPlusCatalogThread type.
IMPORTANT:
In HFS, thread records were required for folders
but optional for files. In HFS Plus, thread records
are required for both files and folders.
struct HFSPlusCatalogThread {
SInt16 recordType;
SInt16 reserved;
HFSCatalogNodeID parentID;
HFSUniStr255 nodeName;
typedef struct HFSPlusCatalogThread HFSPlusCatalogThread;
kHFSPlusFileRecord or
kHFSPlusFolderRecord, depending on whether
the thread record refers to a file or a folder. Both
types of thread record contain the same data.reserved1parentIDnodeNameThe next section explains how thread records can be used to find a file or folder using just its CNID.
File and folder records always have a key that contains a
non-empty nodeName. The file and folder records
for the children are all consecutive in the catalog, since
they all have the same parentID in the key, and
vary only by nodeName.
The key for a thread record is the file's or folder's CNID (not the CNID of the parent folder) and
an empty (zero length) nodeName. This allows a
file or folder to by found using just the CNID. The thread
record contains the parentID and
nodeName field of the file or folder itself.
Finding a file or folder by its CNID is a two-step process. The first step is to use the CNID to look up the thread record for the file or folder. This yields the file or folder's parent folder ID and name. The second step is to use that information to look up the real file or folder record.
Since files do not contain other files or folders, there
are no catalog records whose key has a parentID
equal to a file's CNID and nodeName with
non-zero length. These unused key values are reserved.
See the Finder Interface Reference for more detailed information about these data types and how the Finder uses them.
typedef struct Rect Rect; /* OSType is a 32-bit value made by packing four 1-byte characters together. */ typedef UInt32 FourCharCode; typedef FourCharCode OSType; /* Finder flags (finderFlags, fdFlags and frFlags) */ enum { kIsOnDesk = 0x0001, /* Files and folders (System 6) */ kColor = 0x000E, /* Files and folders */ kIsShared = 0x0040, /* Files only (Applications only) If */ /* clear, the application needs */ /* to write to its resource fork, */ /* and therefore cannot be shared */ /* on a server */ kHasNoINITs = 0x0080, /* Files only (Extensions/Control */ /* Panels only) */ /* This file contains no INIT resource */ kHasBeenInited = 0x0100, /* Files only. Clear if the file */ /* contains desktop database resources */ /* ('BNDL', 'FREF', 'open', 'kind'...) */ /* that have not been added yet. Set */ /* only by the Finder. */ /* Reserved for folders */ kHasCustomIcon = 0x0400, /* Files and folders */ kIsStationery = 0x0800, /* Files only */ kNameLocked = 0x1000, /* Files and folders */ kHasBundle = 0x2000, /* Files only */ kIsInvisible = 0x4000, /* Files and folders */ kIsAlias = 0x8000 /* Files only */ /* Extended flags (extendedFinderFlags, fdXFlags and frXFlags) */ enum { kExtendedFlagsAreInvalid = 0x8000, /* The other extended flags */ /* should be ignored */ kExtendedFlagHasCustomBadge = 0x0100, /* The file or folder has a */ /* badge resource */ kExtendedFlagHasRoutingInfo = 0x0004 /* The file contains routing */ /* info resource */ struct FileInfo { OSType fileType; /* The type of the file */ OSType fileCreator; /* The file's creator */ UInt16 finderFlags; Point location; /* File's location in the folder. */ UInt16 reservedField; typedef struct FileInfo FileInfo; struct ExtendedFileInfo { SInt16 reserved1[4]; UInt16 extendedFinderFlags; SInt16 reserved2; SInt32 putAwayFolderID; typedef struct ExtendedFileInfo ExtendedFileInfo; struct FolderInfo { Rect windowBounds; /* The position and dimension of the */ /* folder's window */ UInt16 finderFlags; Point location; /* Folder's location in the parent */ /* folder. If set to {0, 0}, the Finder */ /* will place the item automatically */ UInt16 reservedField; typedef struct FolderInfo FolderInfo; struct ExtendedFolderInfo { Point scrollPosition; /* Scroll position (for icon views) */ SInt32 reserved1; UInt16 extendedFinderFlags; SInt16 reserved2; SInt32 putAwayFolderID; typedef struct ExtendedFolderInfo ExtendedFolderInfo;HFS Plus tracks which allocation blocks belong to a file's forks by maintaining a list of extents (contiguous allocation blocks) that belong to that file, in the appropriate order. Each extent is represented by a pair of numbers: the first allocation block number of the extent and the number of allocation blocks in the extent. The file record in the catalog B-tree contains a record of the first eight extents of each fork. If there are more than eight extents in a fork, the remaining extents are stored in the extents overflow file.
Note:
Fork Data
Structure
discusses how HFS Plus maintains
information about a fork.
Like the catalog file, the extents overflow file is B-tree . However, the structure of the extents overflow file is relatively simple compared to that of a catalog file. The extents overflow file has a simple, fixed length key and a single type of data record.
The structure of the key for the extents overflow file is
described by the
HFSPlusExtentKey
type.
keyLength
field is required by all
keyed records
in a B-tree.
The extents overflow file, in common with all HFS Plus
B-trees, uses a large key length (
UInt16
).
Keys in the extents overflow file always have the same
length,
kHFSPlusExtentKeyMaximumLength
(10).
forkType
fileID
startBlock
Note:
Typically, an implementation will keep a copy of
the initial extents from the catalog record. When
trying to access part of the fork, they see whether
that position is beyond the extents described in
the catalog record; if so, they use that offset (in
allocation blocks) to find the appropriate extents
B-tree record. See
Extents
Overflow File Usage for more information.
Two
HFSPlusExtentKey
structures are compared
by comparing their fields in the following order:
fileID
,
forkType
,
startBlock
. Thus, all the extent records for a
particular fork are grouped together in the B-tree, right
next to all the extent records for the other fork of the
file.
The data records for an extents overflow file (the
extent records
) are described by the
HFSPlusExtentRecord
type, which is described in
detail in
Fork Data
Structure
.
IMPORTANT:
Remember that the
HFSPlusExtentRecord
contains descriptors for eight extents. The first
eight extents in a fork are held in its
catalog file
record
. So the number of extent records for a
fork is ((number of extents - 8 + 7) / 8).
The most important thing to remember about extents overflow file is that it is only used for forks with more than eight extents. In most cases, forks have fewer extents, and all the extents information for the fork is held in its catalog file record. However, for more fragmented forks, the extra extents information is stored in the extents overflow file.
When an implementation needs to map a fork offset into a position on disk, it first looks through the extent records in the catalog file record. If the fork offset is within one these extents, the implementation can find the corresponding position without consulting the extents overflow file.
If, on the other hand, the fork offset is beyond the last extent recorded in the catalog file record, the implementation must look in the next extent record, which is stored in the extents overflow file. To find this record, the implementation must form a key, which consists of information about the fork (the fork type and the file ID) and the offset info the fork (the start block).
Because extent records are partially keyed off the fork offset of the first extent in the record, the implementation must have all the preceding extent records in order to know the fork offset to form the key of the next extent record. For example, if the fork has two extent records in the extents overflow file, the implementation must read the first extent record to calculate the fork offset for the key for the second extent record.
However, you can use the
startBlock
in the
extent key to go directly to the record you need. Here's a
complicated example:
We've got a fork with a total of 23 extents (very
fragmented!). The
blockCounts
for the extents,
in order, are as follows: one extent of 6 allocation blocks,
14 extents of one allocation block each, two extents of two
allocation blocks each, one extent of 7 allocation blocks,
and five more extents of one allocation block each. The fork
contains a total of 36 allocation blocks.
The block counts for the catalog's fork data are: 6, 1,
1, 1, 1, 1, 1, 1. There is an extent overflow record whose
startBlock is 13 (0+6+1+1+1+1+1+1+1), and has the following
block counts: 1, 1, 1, 1, 1, 1, 1, 2. There is a second
extent overflow record whose
startBlock
is 22
(13+1+1+1+1+1+1+1+2), and has the following block counts: 2,
7, 1, 1, 1, 1, 1, 0. Note this last record only contains
seven extents.
Suppose the allocation block size for the volume is 4K. Suppose we want to start reading from the file at an offset of 108K. We want to know where the data is on the volume, and how much contiguous data is there.
First, we divide 108K (the fork offset) by 4K (the allocation block size) to get 27, which is the number of allocation blocks from the start of the fork. So, we want to know where fork allocation block #27 is. We notice that 27 is greater than or equal to 13 (the number of allocation blocks in the catalog's fork data), so we're going to have to look in the extents B-tree.
We construct a search key with the appropriate
fileID
and
forkType
, and set
startBlock
to 27 (the desired fork allocation
block number). We then search the extents B-tree for the
record whose key is less than or equal to our search key. We
find the second extent overflow record (the one with
startBlock
=22). It has the same
fileID
and
forkType
, so things are
good. Now we just need to figure out which extent within
that record to use.
We compute 27 (the desired fork allocation block) minus
22 (the
startBlock
) and get 5. So, we want the
extent that is 5 allocation blocks "into" the record. We try
the first extent. It's only two allocation blocks long, so
the desired extent is 3 allocation blocks after that first
extent in the record. The next extent is 7 allocation blocks
long. Since 7 is greater than 3, we know the desired fork
position is within this extent (the second extent in the
second overflow record). Further, we know that there are
7-3=4 contiguous allocation blocks (i.e., 16K).
We grab the
startBlock
for that second
extent (i.e., the one whose
blockCount
is 7);
suppose this number is 444. We add 3 (since the desired
position was 3 allocation blocks into the extent we found).
So, the desired position is in allocation block 444+3=447 on
the volume. That is 447*4K=1788K from the start of the HFS
Plus volume. (Since the Volume Header always starts 1K after
the start of the HFS Plus volume, the desired fork position
is 1787K after the start of the Volume Header.)
The extent overflow file is also used to hold information about the bad block file. The bad block file is used to mark areas on the disk as bad, unable to be used for storing data. This is typically used to map out bad sectors on the disk.
Note:
All space on an HFS Plus volume is allocated in
terms of allocation blocks. Typically, allocation
blocks are larger than sectors. If a sector is
found to be bad, the entire allocation block is
unusable.
When an HFS Plus volume is embedded within an HFS wrapper (the way Mac OS normally initializes a hard disk), the space used by the HFS Plus volume is marked as part of the bad block file within the HFS wrapper itself . (This sounds confusing because you have a volume within another volume.)
The bad block file is not a file in the same sense as a
user file (it doesn't have a file record in the catalog) or
one of the special files (it's not referenced by the volume
header). Instead, the bad block file uses a special
(
kHFSBadBlockFileID
) as the key for extent
records in the extents overflow file. When a block is marked
as bad, an extent with this CNID and encompassing the bad
block is added to the extents overflow file. The block is
marked as used in the
allocation
file
. These steps prevent the block from being used for
data by the file system.
Bad block extent records are always assumed to reference
the data fork. The
forkType
field of the key
must be 0.
Note:
Because an extent record holds up to eight extents,
adding a bad block extent to the bad block file
does not necessarily require the addition of a new
extent record.
HFS uses a similar mechanism to store information about bad blocks. This facility is used by the HFS Wrapper to hold an entire HFS Plus volume as bad blocks on an HFS disk.
HFS Plus uses an allocation file to keep track of whether each allocation block in a volume is currently allocated to some file system structure or not. The contents of the allocation file is a bitmap. The bitmap contains one bit for each allocation block in the volume. If a bit is set, the corresponding allocation block is currently in use by some file system structure. If a bit is clear, the corresponding allocation block is not currently in use, and is available for allocation.
Note:
HFS stores allocation information in a special area
on the volume, known as the
volume bitmap
.
The allocation file mechanism used by HFS Plus has
a number of advantages.
Each byte in the allocation file holds the state of eight allocation blocks. The byte at offset X into the file contains the allocation state of allocations blocks (X * 8) through (X * 8 + 7). Within each byte, the most significant bit holds information about the allocation block with the lowest number, the least significant bit holds information about the allocation block with the highest number. Listing 1 shows how you would test whether an allocation block is in use, assuming that you've read the entire allocation file into memory.
static Boolean IsAllocationBlockUsed(UInt32 thisAllocationBlock, UInt8 *allocationFileContents) UInt8 thisByte; thisByte = allocationFileContents[thisAllocationBlock / 8]; return (thisByte & (1 << (7 - (thisAllocationBlock % 8)))) != 0;Listing 1 Determining whether an allocation block is in use.
The size of the allocation file depends on the number of allocation blocks in the volume, which in turn depends both on the size of the disk and on the size of the volume's allocation blocks. For example, a volume on a 1 GB disk and having an allocation block size of 4 KB needs an allocation file size of 256 Kbits (32 KB, or 8 allocation blocks). Since the allocation file itself is allocated using allocation blocks, it always occupies an integral number of allocation blocks (its size may be rounded up).
The allocation file may be larger than the minimum number of bits required for the given volume size. Any unused bits in the bitmap must be set to zero.
Note:
Because the entire volume is composed of allocation blocks (with the possible exception of the alternate volume header, as described above), the volume header, alternate volume header, and reserved areas (the first 1024 bytes and the last 512 bytes) must be marked as allocated in the allocation file. The actual number of allocation blocks allocated for these areas varies with the size of the allocation blocks. Any allocation block that contains any of these areas must be marked allocated.
Since the number of allocation blocks is determined by a 32-bit number, the size of the allocation file can be up to 512 MB in size, a radical increase over HFS's 8 KB limit.For example, if 512-byte allocation blocks are used, the first three and last two allocation blocks are allocated. With 1024-byte allocation blocks, the first two and the last allocation blocks are allocated. For larger allocation block sizes, only the first and last allocation blocks are allocated for these areas.
See the Volume Header section for a description of these areas.
Attributes File
The HFS Plus attributes file is reserved for implementing named forks in the future. An attributes file is organized as a B-tree file. It a special file, described by an
HFSPlusForkDatarecord in the volume header, with no entry in the catalog file. An attributes files has a variable length key and three data record types, which makes it roughly as complex as the catalog file.It is possible for a volume to have no attributes file. If the first extent of the attributes file (stored in the volume header) has zero allocation blocks, the attributes file does not exist.
The B-Trees chapter defined a standard rule for the node size of HFS Plus B-trees. As the attributes file is a B-tree, it inherits the requirements of this rule. In addition, the node size of the attributes file must be at least 4 KB (
The exact organization of the attributes B-tree has not been fully designed. Specifically:kHFSPlusAttrMinNodeSize).
An implementation written to this specification may use the details that are final to perform basic consistency checks on attributes. These checks will be compatible with future implementations written to a final attributes specification. See Attributes and the Allocation File Consistency Check.
The leaf nodes of an attributes file contain data records, known as attributes. There are two types of attributes:
Each record starts with a recordType field,
which describes the type of attribute data record. The
recordType field contains one of the following
values.
HFSPlusAttrForkData type to interpret the
data.kHFSPlusAttrExtentsHFSPlusAttrExtents type to
interpret the data. A record of type
kHFSPlusAttrExtents is really just overflow
extents for a corresponding record of type
kHFSPlusAttrForkData. (Think of
kHFSPlusAttrForkData as being like a catalog
record and kHFSPlusAttrExtents as being like
an extents overflow record.)The next two sections describe the fork data and extension attributes in detail.
A fork data attribute is defined by the
HFSPlusAttrForkData data type.
kHFSPlusAttrForkData.reservedtheForkHFSPlusForkData type.A extension attribute is defined by the
HFSPlusAttrExtents data type.
struct HFSPlusAttrExtents {
UInt32 recordType;
UInt32 reserved;
HFSPlusExtentRecord extents;
typedef struct HFSPlusAttrExtents HFSPlusAttrExtents;
kHFSPlusAttrExtents.reservedextentsHFSPlusExtentRecord type.While the key structure for the attributes file is not fully specified, it is still possible for an implementation to use attribute file information in its allocation file consistency check. The leaf records of the attribute file are fully defined, so the implementation can simply iterate over them to determine which allocation blocks on the disk are being used by fork data attributes.
See Allocation File Consistency Check for details.
The startup file is a special file intended to hold information needed when booting a system that does not have built-in (ROM) support for HFS Plus. A boot loader can find the startup file without full knowledge of the HFS Plus volume format (B-trees, catalog file, and so on). Instead, the volume header contains the location of the first eight extents of the startup file.
It is legal for the startup file to contain more than eight extents, and for the remaining extents to be placed in the extents overflow file. However, doing so defeats the purpose of the startup file.Note:
Mac OS does not use the startup file to boot from HFS Plus
disks. Instead, it uses the HFS wrapper, as
described later in this document.
Hard links are a feature that allows multiple directory entries to refer to a single file's content. They are a way to give a single file multiple names, possibly in multiple directories. This section describes how Mac OS X implements hard links on HFS Plus volumes.
The Mac OS X implementation of hard links on HFS Plus volumes was done using the existing metadata fields of the catalog records. This makes it possible to back up and restore a volume using hard links, by backing up and restoring individual files, without having to understand or interpret the hard links. An HFS Plus implementation may choose to automatically follow hard links, or not.
Hard links in HFS Plus are represented by a set of several files. The actual file content (which is shared by each of the hard links) is stored in a special indirect node file. This indirect node file is the equivalent of an inode in a traditional UNIX file system.
HFS Plus uses special hard link files (or links) to refer (or point) to an indirect node file. There is one hard link file for each directory entry or name that refers to the file content.
Indirect node files exist in a special directory called the
metadata directory. This directory exists in the volume's root
directory. The name of the metadata directory is four null
characters followed by the string "HFS+ Private Data". The
directory's creation date is set to the creation date of the
volume's root directory. The kIsInvisible and
kNameLocked bits are set in the directory's
Finder information. The icon
location in the Finder info is set to the point
(22460, 22460). These Finder info settings are not mandatory,
but they tend to reduce accidental changes to the metadata directory.
An implementation that automatically follows hard links should
make the metadata directory inaccessable from its normal
file system interface.
Note:
The case-insensitive Unicode
string comparison used by HFS Plus and case-insensitive
HFSX sorts null characters after all other
characters, so the metadata directory will typically be the last
item in the root directory. On case-sensitive HFSX
volumes, null characters sort before other characters, so the
metadata directory will typically be the first item in the root directory.
Indirect node files have a special identifying number called a link reference. The link reference is unique among indirect node files on a given volume. The link reference is not related to catalog node IDs. When a new indirect node file is created, it is assigned a new link reference randomly chosen from the range 100 to 1073741923.
The file name of an indirect node file is the string "iNode" immediately followed by the link reference converted to decimal text, with no leading zeroes. For example, an indirect node file with link reference 123 would have the name "iNode123".
An indirect node file must be a file, not a directory. Hard links to directories are not allowed because they could cause cycles in the directory hierarchy if a hard link pointed to one of its ancestor directories.
The linkCount field in the
permissions is an estimate of
the number of links referring to this indirect node file. An
implementation that understands hard links should increment this
value when creating an additional link, and decrement the value
when removing a link. However, some implementations (such as
traditional Mac OS) do not understand hard links and may make
changes that cause the linkCount to be inaccurate.
Similarly, it is possible for a link to refer to an indirect
node file that does not exist. When removing a link, an
implementation should not allow the linkCount
to underflow; if it is already zero, do not change it.
Note:
The inode number returned by the POSIX stat
or lstat routines in the st_ino
field of the stat structure is actually the
catalog node ID of the indirect
node file, not the link reference mentioned above.
The reason for using a separate link reference number, instead of a catalog node ID, is to allow hard links to be backed up and restored by utilities that are not specifically aware of hard links. As long as they preserve filenames, Finder info, and permissions, then the hard links will be preserved.
Hard link files are ordinary files in the catalog. The catalog node ID of a hard link file is different from the catalog node ID of the indirect node file it refers to, and different from the catalog node ID of any other hard link file.
The fileType and fileCreator fields
of the userInfo in the
catalog record of a hard link file must be set to
kHardLinkFileType and kHFSPlusCreator,
respectively. The hard link file's creation date should be set to
the creation date of the metadata directory. The hard link file's
creation date may also be set to the creation date of the volume's
root directory (if it differs from the creation date of the metadata
directory), though this is deprecated. The iNodeNum field
in the permissions is set to the
link reference of the indirect node file that the link refers to.
For better compatibility with older versions of the Mac OS Finder,
the kHasBeenInited flag should be set in the Finder
flags. The other Finder information, and other dates in the catalog
record are reserved.
enum {
kHardLinkFileType = 0x686C6E6B, /* 'hlnk' */
kHFSPlusCreator = 0x6866732B /* 'hfs+' */
POSIX semantics allow an open file to
be unlinked (deleted). These open but unlinked files are stored on HFS
Plus volumes much like a hard link. When the open file is deleted, it
is renamed and moved into the metadata directory. The new name is the
string "temp" followed by the catalog node ID
converted to decimal text. When the file is eventually closed, this
temporary file may be removed. All such temporary files may be removed
when repairing an unmounted HFS Plus volume.
Repairing the Metadata Directory
When repairing an HFS Plus volume with hard links or a metadata
directory, there are several conditions that might need to be repaired:
Opened but deleted files are files whose names start with "temp", and are in the metadata directory. If the volume is not in use (not mounted, and not being used by any other utility), then these files can be deleted. Volumes with a journal, even one with no active transactions, may have opened but undeleted files that need to be deleted.
Detecting an orphaned indirect node file, broken hard link, or incorrect link count requires finding all hard link files in the catalog, and comparing the number of found hard links for each link reference with the link count of the corresponding indirect node file.
A hard link with a link reference equal to 0 is invalid. Such a hard link may be the result of a hard link being copied or restored by an implementation or utility that does not use the permissions in catalog records. It may be possible to repair the hard link by determining the proper link reference. Otherwise, the hard link should be deleted.
Similar to a hard link, a symbolic link is a special kind of file that refers to another file or directory. A symbolic link stores the path name of the file or directory it refers to.
On an HFS Plus volume, a symbolic link is stored
as an ordinary file with special values in some of the fields of its
catalog record. The pathname of the
file being referred to is stored in the data fork. The file type in
the fileMode field of the
permissions is set to S_IFLNK. For compatibility
with Carbon and Classic applications, the file type of a symbolic
link is set to kSymLinkFileType, and the creator code
is set to kSymLinkCreator. The resource fork of the
symbolic link has zero length and is
reserved.
enum {
kSymLinkFileType = 0x736C6E6B, /* 'slnk' */
kSymLinkCreator = 0x72686170 /* 'rhap' */
Note:
The pathname stored in a symbolic link is assumed to be a POSIX
pathname, as used by the Mac OS X BSD and Cocoa programming interfaces.
It is not a traditional Mac OS, or Carbon, pathname. The path
is encoded in UTF-8. It must be a valid UTF-8 sequence, with no null
(zero) bytes. The path may refer to another volume. The path need
not refer to any existing file or directory. The path may be full
or partial (with or without a leading forward slash). For maximum
compatibility, the length of the path should be 1024 bytes or less.
Journal
An HFS Plus volume may have an optional journal to speed
recovery when mounting a volume that was not unmounted safely
(for example, as the result of a power outage or crash). The
journal makes it quick and easy to restore the volume
structures to a consistent state, without having to scan all of
the structures. The journal is used only for the volume
structures and metadata; it does not protect the contents of a
fork.
Background
A single change to the volume may require writing coordinated
changes to many different places on the volume. If a failure
happens after some, but not all, of the changes have been
written, then the volume may be seriously damaged and may result
in catastrophic loss of data.
For example, creating a file or directory requires adding two
records (the file or folder record, and its thread record) to the
catalog B-tree. A leaf node may not have enough room for a new
record, so it may have to be split. That means that some records
will be removed from the existing node and put into a newly
allocated node. This requires adding a new key and pointer to
the index node that is the parent of the leaf being split (which
might require splitting the index node, and so on). If a failure
occurs after the split, but before the index node is updated,
all of the items in the new node would become inaccessible.
Recovery from this sort of damage is time consuming at best,
and may be impossible.
The purpose of the journal is to ensure that when a group of
related changes are being made, that either all of those changes
are actually made, or none of them are made. This is done by
gathering up all of the changes, and storing them in a separate
place (in the journal). Once the journal copy of the changes
is completely written to disk, the changes can actually be
written to their normal locations on disk. If a failure happens
at that time, the changes can simply be copied from the
journal to their normal locations. If a failure happens when
the changes are being written to the journal, but before they are
marked complete, then all of those changes are ignored.
A group of related changes is called a transaction.
When all of the changes of a transaction have been written to
their normal locations on disk, that transaction has been
committed, and is removed from the journal. The journal
may contain several transactions. Copying changes from all
transactions to their normal locations on disk is called
replaying the journal.
IMPORTANT:
Implementations accessing a journaled volume with transactions
must either refuse to access the volume, or replay the journal
to be sure the volume is consistent. If the
lastModifiedVersion field of the
volume header does not match the
signature of an implementation known to properly use and update
the journal, then the journal must not be replayed (since it may
no longer match the on-disk structures, and could cause
corruption if replayed).
Overview of Journal Data Structures
If kHFSVolumeJournaledBit is set in the volume header's attributes field, the
volume has a journal. The journal data stuctures consist of a
journal info block, journal header, and journal buffer. The
journal info block indicates the location and size of the
journal header and journal buffer. The journal buffer is
the space set aside to hold transactions. The journal
header describes which part of the journal buffer is active
and contains transactions waiting to be committed.
Figure 7. Overview of an HFS Plus Journal.
On HFS Plus volumes, the journal info block is stored as a
file (so that its space can be properly represented in a catalog
record and the allocation bitmap). The name of that file is
".journal_info_block" and it is stored in the
volume's root directory. The journal header and journal buffer
are stored together in a different file named
".journal", also in the volume's root directory.
Each of these files are contiguous on disk (they occupy exactly
one extent). An implementation that uses the journal must
prevent the files from being accessed as normal files.
Note:
An implementation must find the journal info block by using the
journalInfoBlock field of the volume header, not by
the file name. Similarly, an implementation must find the
journal header and journal buffer using the contents of the
journal info block, not the file name.
A single transaction consists of several blocks, including
both the data to be written, and the location where that data is
to be written. This is represented on disk by a block list
header, which describes the number and sizes of the blocks,
immediately followed by the contents of those blocks.
Figure 8. A Simple Transaction.
Since block list headers are of limited size, a single
transaction may consist of several block list headers and their
associated block contents (one block list header followed by the
contents of the blocks that header describes, then the next
block list header and its block contents, and so on). If the
next field of the first block_info is
non-zero, then the next block list header is a continuation of
the same transaction.
Figure 9. A Transaction with Multiple Block Lists.
The journal buffer is treated as a circular buffer. When
reading or writing the journal buffer, the I/O operation must
stop at the end of the journal buffer and resume (wrap around)
immediately following the journal header. Block list headers or
the contents of blocks may wrap around in this way. Only a
portion of the journal buffer is active at any given time; this
portion is indicated by the start and
end fields of the journal header. The part of the
journal buffer that is not active contains no meaningful data,
and must be ignored.
To prevent ambiguity when start equals
end, the journal is never allowed to be perfectly
full (all of the journal buffer used by block lists and blocks).
If the journal was perfectly full, and start was
not equal to jhdr_size, then end would
be equal to start. You would then be unable to
differentiate between an empty and full journal.
When the journal is not empty (contains transactions),
it must be replayed to be sure the volume
is consistent. That is, the data from each of the transactions must be
written to the correct blocks on disk.
Journal Info Block
The journal info block describes where the journal header and
journal buffer are stored. The journal info block is stored at the
beginning of the allocation block whose number is stored in the
journalInfoBlock field of the
volume header. The journal info block
is described by the data type JournalInfoBlock.
device_signaturekJIJournalOnOtherDeviceMask is set).offsetkJIJournalInFSMask
is set), this offset is relative to the start of the volume.reservedThe following constants define bit flags in the flags field:
kJIJournalInFSMask = 0x00000001,
kJIJournalOnOtherDeviceMask = 0x00000002,
kJIJournalNeedInitMask = 0x00000004
offset field of the journal info block
is relative to the start of the volume (allocation
block number zero).kJIJournalOnOtherDeviceMaskdevice_signature field
in the journal info block describes the device containing
the journal.kJIJournalNeedInitMaskkJIJournalInFSMask, but not
kJIJournalOnOtherDeviceMask. Journals stored on a
separate device are not currently supported. The format of the
device_signature field is not yet defined.
The journal begins with a journal header, whose main purpose
is to describe the location of transactions in the journal
buffer. The journal header is stored using the
journal_header data type.
JOURNAL_HEADER_MAGIC (0x4a4e4c78).
This is used to verify the integrity of the journal header.endianENDIAN_MAGIC (0x12345678).
This is used to verify the integrity of the journal header.startstart field,
indicating that the transactions wrap around the end of the
journal's circular buffer. If end equals
start, then the journal is empty, and there are
no transactions that need to be replayed.size field of the
journal info block.blhdr_sizechecksumjhdr_sizeThe block list header describes a list of blocks
included in a transaction. A transaction may include several
block lists if it modifies more blocks than can be represented
in a single block list. The block list header is stored in a
structure of type block_list_header.
block_info
items) this block list can describe. This field is used
while in memory to keep track of journal buffer sizes. On
disk, this field is reserved.num_blocksbinfo array. Since
the first element of the binfo array is used to
chain multiple block lists into a single transaction, the
actual number of data blocks is num_blocks - 1.
bytes_usedbytes_used bytes from the start of the current block
list header, wrapping from the end of the journal buffer to the
start of the journal buffer if needed.checksumbinfo array
(a total of 32 bytes).binfonum_blocks+1 entries. The first entry is used
when a single transaction occupies multiple block lists,
using the next field as described above. The
remaining num_blocks entries describe where the
data from this block list will be written to disk.The first element of the binfo array is
used to indicate whether the transaction contains additional
block lists. Each of the other elements of the
binfo array represent a single block of data
in the journal buffer which must be copied to its correct
location on disk. The fields have the following meaning:
binfo array.bsizebinfo array.block_info of a block
list, then the transaction ends with this block list;
otherwise, the transaction has one or more additional block
lists. This field is meaningful only for the first element
of the block list array. The actual on-disk value has no
meaning beyond testing for zero or non-zero.The journal header and
block list header both contain
checksum fields. These checksums can be verified as part of a
basic consistency check of these structures. To verify the
checksum, temporarily set the checksum field to zero and then call
the calc_checksum routine with the address and size of the
header being checksummed. The function result should equal the
original value of the checksum field.
In order to replay the journal, an implementation just loops over the transactions, copying each individual block in the transaction from the journal to its proper location on the volume. Once those blocks have been flushed to the media (not just the driver!), it may update the journal header to remove the transactions.
Note:
Replaying the journal does not guarantee that the temporary files
associated with open but unlinked
files are deleted. After replaying the journal, these temporary
files may be deleted.
Here are the steps to replay the journal:
vhb. The volume may have an
HFS wrapper; if so, you will need to use
it to determine the location of the volume header.kHFSVolumeJournaledBit in the
attributes field of the volume header. If
it is not set, there is no journal to replay, and you
are done.vhb.journalInfoBlock,
into variable jib.kJIJournalNeedsInitMask is set in jib.flags,
the journal was never initialized, so there is no journal to replay.kJIJournalInFSMask is set and
kJIJournalOnOtherDeviceMask is clear in
jib.flags.jib.offset bytes from the start of the volume, and
place it in variable jhdr.jhdr.start equals jhdr.end, the
journal does not have any transactions, so there is nothing
to replay.jhdr.start.jhdr.start does not equal jhdr.end,
perform the following steps:
jhdr.blhdr_size bytes from the current offset
in the journal into variable blhdr.bhdr.binfo[1] to
bhdr.binfo[blhdr.num_blocks], inclusive, copy
bsize bytes from the current offset in the
journal to sector bnum on the volume (to byte
offset bnum*jdhr.jhdr_size). Remember that
jhdr_size is the size of a sector, in bytes.bhdr.binfo[0].next is zero, you have completed
the last block list of the current transaction; set
jhdr.start to the current offset in the journal.Note:
Remember that the journal is a circular buffer. When reading a
block list header or block from
the journal buffer (in the loop described above), you will need
to check whether it wraps around the end of the journal buffer.
If it would extend beyond the end of the journal buffer, you
must stop reading at the end of the journal buffer, and resume
reading at the start of the journal buffer (offset
jhdr.jhdr_size bytes from the start of the
journal).
After replaying an entire transaction (all blocks
in a block list, when bhdr.binfo[0] is zero), or after
replaying all transactions, you may update the value of the start
field in the journal header to the current
offset in the journal. This will remove those block lists from the
journal since they have been written to their correct locations on disk.
WARNING:
You must ensure that previous block writes complete before updating
the journal header's start field on disk. One way to
do this is to issue a flush to the device driver and wait until
the device driver has written all dirty blocks, and then flush the
device itself and wait for the device to write all dirty blocks
to the media.
HFSX is an extension to HFS Plus to allow additional features that are incompatible with HFS Plus. The only such feature currently defined is case-sensitive filenames.
HFSX volumes have a signature of 'HX'
(0x4858) in the signature field of the
volume header. The version
field identifies the version of HFSX used on the volume; the only
value currently defined is 5. If features are added that
would be incompatible with older versions (that is, older versions
cannot safely access or modify the volume because of the new features),
then a different version number will be used.
Note:
A new signature was required because some utilities
did not use the version field properly. They
would attempt to use or repair the volume (including changing the
version field) when they encountered a
version value that was not previously documented.
WARNING:
If your implementation encounters an HFSX volume with a
version value it does not recognize, it must
not attempt to access or repair the volume. Catastrophic
data loss may result. In particular, do NOT change the
version field.
It is intended that future HFSX features will result in the definition of new volume attribute bits, and that those bits will be used to indicate which features are in use on the volume.
An HFSX volume never has an HFS wrapper.
In an Apple partition map, the partition type (PMPartType)
of an HFSX volume is set to "Apple_HFSX".
Introduced in Mac OS X 10.3, HFSX version 5 allows volumes with case-sensitive file and directory names. Case-sensitive names means that you can have two objects, whose names differ only by the case of the letters, in the same directory at the same time. For example, you could have "Bob", "BOB", and "bob" in the same directory.
An HFSX volume may be either case-sensitive or case-insensitive.
Case sensitivity (or lack thereof) is global to the volume; the
setting applies to all file and directory names on the volume.
To determine whether an HFSX volume is case-sensitive, use the
keyCompareType field of the
B-tree header of the catalog file.
A value of kHFSBinaryCompare means the volume is
case-sensitive. A value of kHFSCaseFolding means the
volume is case-insensitive.
Note:
Do not assume that an HFSX volume is case-sensitive.
Always use the keyCompareType to determine
case-sensitivity or case-insensitivity.
A case-insensitive HFSX volume (one whose keyCompareType
is kHFSCaseFolding uses the same
Unicode string comparison algorithm as HFS Plus.
A case-sensitive HFSX volume (one whose keyCompareType
is kHFSBinaryCompare) simply compares each character of
the name as an unsigned 16-bit integer. The first character (the one
with the smallest offset from the start of the string) that is
different determines the relative order. The string with the
numerically smaller character value is ordered before the string
with the larger character value. For example, the string "Bob"
would sort before the string "apple" because the code for the
character "B" (decimal 66) is less than the code for the character
"a" (decimal 97).
IMPORTANT:
Case-sensitive names do not ignore Unicode "ignorable"
characters. This means that a single directory may have several
names which would be considered equivalent using Unicode comparison
rules, but which are considered distinct on a case-sensitive
HFSX volume.
Note:
The null character (0x0000), as used in the name
of the "HFS+ Private Data" directory used by
hard links, sort first with
case-sensitive compares, but last with case-insensitive
compares.
Mac OS X version 10.3 introduced a new policy for determining where to allocate space for files, which improves performance for most users. This policy places the volume metadata and frequently used small files ("hot files") near each other on disk, which reduces the seek time for typical accesses. This area on disk is known as the metadata zone.
The volume metadata are the structures that let the file system manage the contents of the volume. It includes the allocation bitmap file, extents overflow file, and the catalog file, and the journal file. The volume header and alternate volume header are also metadata, but they have fixed locations within the volume, so they are not located in the hot file area. Mac OS X may use a quota users file and quota groups file to manage disk space quotas on a volume. These files aren't strictly metadata, but they are included in the metadata zone because of their heavy use by the OS and they are too large to be considered ordinary hot files.
Implementations are encouraged not to interfere with the metadata zone policy. For example, a disk optimizer should avoid moving files into the metadata zone unless that file is known to be frequently accessed, in which case it may be added to the "hot file" list. Similarly, files in the metadata zone should not be moved elsewhere on disk unless they are also removed from the hot file list.
This policy is only applied to volumes whose size is at least 10GB, and which have journaling enabled. The metadata zone is established when the volume is mounted. The size of the zone is based upon the following sizes:
Contribution to the Metadata Zone size Allocation Bitmap File Physical size (totalBlocks times the volume's
allocation block size) of the allocation bitmap file.
Extents Overflow File
4MB, plus 4MB per 100GB (up to 128MB maximum)
Journal File
8MB, plus 8MB per 100GB (up to 512MB maximum)
Catalog File
10 bytes per KB (1GB minimum)
Hot Files
5 bytes per KB (10MB minimum; 512MB maximum)
Quota Users File
Described below
Quota Groups File
Described below
In Mac OS X version 10.3, the amount of space reserved for the allocation file is actually the minimum allocation file size for the volume (the total number of allocation blocks, divided by 8, rounded up to a multiple of the allocation block size). If the allocation file is larger than that (which is sometimes done to allow a volume to be more easily grown at a later time), then there will be less space available for other metadata or hot files in the metadata zone. This is a bug (r. 3522516).
The amount of space reserved for each type of metadata (except for the allocation bitmap file) is based on the total size of the volume. For the purposes of these computations, the total size of the volume is the allocation block size multiplied by the total number of allocation blocks.
The sizes reserved for quota users and groups files are the result of
complex calculations. In each case, the size reserved is a value of
the form (items + 1) * 64 bytes, where items
is based on the size of the volume in gigabytes, rounded down. For the
quota users file, items is 256 per gigabyte, rounded up to
a power of 2, with a minimum of 2048, and a maximum of 2097152 (2M).
For the quota groups file, items is 32 per gigabyte,
rounded up to a power of 2, with a minimum of 2048, and a maximum of
262144 (256K). The quota files are considered hot files, and occupy
the hot file area, even though they are larger
than the maximum file size normally eligible to be a hot file.
The total size of the metadata zone is the sum of the above sizes, rounded up so that the metadata zone is represented by a whole number of allocation blocks within the volume bitmap. That is, the start and end of the metadata zone fall on allocation block boundaries in the volume bitmap. That means that the size of the metadata zone is rounded up to a multiple of 8 times the square of the allocation block size. In Mac OS X version 10.3, the extra space due to the round up of the metadata zone is split up between the catalog and the hot file area (2/3 and 1/3, respectively).
The calculations for the extents overflow file and journal file divide the total size of the volume by 100GB, rounding down. Then they add one (to compensate for any remainder lost as part of the rounding). The result is then multiplied by 4MB or 8MB, respectively. If the volume's total size is not a multiple of 100GB, this is equivalent to 4MB (or 8MB) per 100GB, rounded up.
In Mac OS X version 10.3, the metadata zone is located at the start of the volume, following the volume header. The hot file area is located towards the end of the metadata zone.
When performing normal file allocations, the allocator will skip over the metadata zone. This ensures that the metadata will be less fragmented, and all of the metadata will be located in the same area on the disk. If the area outside the metadata zone is exhausted, the allocator will then use space inside the metadata zone for normal file allocations. Similarly, when allocating space for metadata, the allocator will use space inside the metadata zone first. If all of the metadata zone is in use, then metadata allocations will use space outside the metadata zone.
Most files on a disk are rarely, if ever, accessed. Most frequently accessed (hot) files are small. To improve performance of these small, frequently access files, they are moved near the volume's metadata, into the metadata zone. This reduces seek times for most accesses. As files are moved into the metadata zone, they are also defragmented (allocated in a single extent), which further improves performance. This process is known as adaptive hot file clustering.
The relative importance of a frequently used (hot) file is called its temperature. Files with the hottest (largest) temperatures are the ones actually moved into the metadata zone. In Mac OS X version 10.3, a file's temperature is computed as the number of bytes read from the file during the recording period divided by the file's size in bytes. This is a measure of how often the file is read.
This section describes the on-disk structures used for tracking hot files. The algorithms used at run time are subject to change, and are not documented here.
Migration of files into or out of the hot file area of the metadata zone is a gradual process, based upon the user's actual file access patterns. The migration happens in several phases:
A B-Tree is used to keep track of the
files that currently occupy the hot file area of the metadata zone. The hot file B-tree is an
ordinary file on the volume (that is, it has records in the catalog). It is a file named
".hotfiles.btree" in the root directory. To avoid
accidental manipulation of this file, the
kIsInvisible and kNameLocked bits in
the finderFlags field of the Finder info should be set.
The node size of the hot file B-tree is at least 512 bytes,
and is typically the same as the the volume's allocation block
size. Like other B-trees on an HFS Plus volume, the key length
field is 16 bits, and kBTBigKeysMask is set in the
B-tree header's attributes. The
btreeType in the header
record must be set to kUserBTreeType.
The B-tree's user data record
contains information about hot file recording. The format of the user
data is described by the HotFilesInfo structure:
#define HFC_MAGIC 0xFF28FF26
#define HFC_VERSION 1
#define HFC_DEFAULT_DURATION (3600 * 60)
#define HFC_MINIMUM_TEMPERATURE 16
#define HFC_MAXIMUM_FILESIZE (10 * 1024 * 1024)
char hfc_tag[] = "CLUSTERED HOT FILES B-TREE ";
struct HotFilesInfo {
UInt32 magic;
UInt32 version;
UInt32 duration; /* duration of sample period */
UInt32 timebase; /* recording period start time */
UInt32 timeleft; /* recording period stop time */
UInt32 threshold;
UInt32 maxfileblks;
UInt32 maxfilecnt;
UInt8 tag[32];
typedef struct HotFilesInfo HotFilesInfo;
versionHotFilesInfo
structure. Version 1 of the structure is described here.
If your implementation encounters any other version number,
it should not read or modify the hot file B-tree.durationHFC_DEFAULT_DURATION
(60 hours).timebasetimeleftthresholdmaxfileblksHFC_MAXIMUM_FILESIZE divided by the volume's
allocation block size.maxfilecnt"CLUSTERED HOT FILES B-TREE " (not including the
quotes). Note that the last six bytes are five spaces and the
null (zero) byte. This field exists to make it easier to recognize
the hot file B-tree when debugging or using a disk editor. An
implementation should not attempt to verify or change this field.A key in the hot file B-tree is of type HotFileKey.
keyLengthkeyLength
field itself. Hot file keys are of fixed size. This field must contain
the value 10.forkType0x00) or a resource fork (value 0xFF).
In Mac OS X version 10.3, only data forks are eligible for placement
into the hot file area.temperatureHFC_LOOKUPTAG
(0xFFFFFFFF).fileIDHot file keys are compared first by temperature,
then fileID, and lastly by forkType.
All of these comparisons are unsigned.
Much like the catalog file, the hot file B-tree stores two kinds of records: hot file records and thread records. Every fork in the hot file area has both a hot file record and a thread record in the hot file B-tree. Hot file records are used to find hot files based on their temperature. Thread records are used to find hot files based on their catalog node ID and fork type.
Thread records in the hot file B-tree use a special value
(HFC_LOOKUPTAG) in the temperature
field of the key. The data for a thread record is the
temperature of that fork, stored as a
UInt32. So, given a catalog node
ID and fork type, it is possible to construct a key for the
fork's thread record. If a thread record exists, you can get
the temperature from the thread's data to construct the key for
the hot file record. If a thread record does not exist, then
the fork is not being tracked as a hot file.
Hot file records use all of the key fields as described above. The data for a hot file record is 4 bytes. The data in a hot file record is not meaningful. To aid in debugging, Mac OS X version 10.3 typically stores the first four bytes of the file name (encoded in UTF-8), or the ASCII text "????".
When an implementation changes a hot file's temperature, the old hot file record must be removed, a new hot file with the new temperature must be inserted, and the thread record's data must be changed to contain the new temperature.
The recording phase gathers information about file usage over time. In order to gather useful statistics, the recording phase may last longer than the duration of a single mount. Therefore, information about file usage is stored on disk so that it can accumulate over time.
The clumpSize field of the
fork data structure is used to record the amount of data actually
read from a fork. Since the field is only 32 bits long, it stores
the number of allocation blocks read from the file. The fork's
temperature can be computed by dividing its clumpSize by
its totalBlocks.
HFS Plus makes heavy use of Unicode strings to store file and folder names. However, Unicode is still evolving, and its use within a file system presents a number of challenges. This section describes some of the challenges, along with the solutions used by HFS Plus.
IMPORTANT:
Before reading this section, you should read
HFS Plus Names.
Note:
The Mac OS Text Encoding Converter provides several
constants that let you convert to and from the canonical,
decomposed form stored on HFS Plus volumes. When using
CreateTextEncoding to create a text encoding,
you should set the TextEncodingBase to
kTextEncodingUnicodeV2_0, set the
TextEncodingVariant to
kUnicodeCanonicalDecompVariant, and set the
TextEncodingFormat to
kUnicode16BitFormat. Using these values ensures
that the Unicode will be in the same form as on an HFS Plus
volume, even as the Unicode standard evolves.
Unicode allows some sequences of characters to be
represented by multiple, equivalent forms. For example, the
character "
"
can be represented as the single Unicode character
u+00E9 (latin small letter e with acute), or as
the two Unicode characters u+0065 and u+0301 (the letter "e"
plus a combining acute symbol).
To reduce complexity in the B-tree key comparison routines (which have to compare Unicode strings), HFS Plus defines that Unicode strings will be stored in fully decomposed form, with composing characters stored in canonical order. The other equivalent forms are illegal in HFS Plus strings. An implementation must convert these equivalent forms to the fully decomposed form before storing the string on disk.
The Unicode Decomposition table contains a list of characters that are illegal as part of an HFS Plus string, and the equivalent character(s) that must be used instead. Any character appearing in a column titled "Illegal", must be replaced by the character(s) in the column immediately to the right (titled "Replace With").
Note:
Mac OS versions 8.1 through 10.2.x used decompositions based on
Unicode 2.1. Mac OS X version 10.3 and later use decompositions
based on Unicode 3.2. Most of the characters whose decomposition
changed are not used by any Mac encoding, so they are unlikely to
occur on an HFS Plus volume. The MacGreek encoding had the largest
number of decomposition changes.
The Unicode decomposition table mentioned above indicates which decompositions were added, removed, or changed between Unicode 2.1 and Unicode 3.2.
In addition, the Korean Hangul characters with codes in the range u+AC00 through u+D7A3 are illegal and must be replaced with the equivalent sequence of conjoining jamos, as described in the Unicode 2.0 book, section 3.10.
The characters with codes in the range u+2000 through u+2FFF are punctuation, symbols, dingbats, arrows, box drawing, etc. The u+24xx block, for example, has single characters for things like "(a)". The characters in this range are not fully decomposed; they are left unchanged in HFS Plus strings. This allows strings in Mac OS encodings to be converted to Unicode and back without loss of information. This is not unnatural since a user would not necessarily expect a dingbat "(a)" to be equivalent to the three character sequence "(", "a", ")" in a file name.The characters in the range u+F900 through u+FAFF are CJK compatibility ideographs, and are not decomposed in HFS Plus strings.
So, for the example given earlier, "
"
must be stored as the two Unicode characters u+0065 and
u+0301 (in that order). The Unicode character u+00E9 may not
appear in a Unicode string used as part of an HFS Plus
B-tree key.
In HFS Plus and case-insensitive HFSX, strings must be compared in a case-insensitive fashion. The Unicode standard does not strictly define upper and lower case equivalence, although it does suggest some equivalences. The HFS Plus string comparison algorithm (defined below) include a concrete case equivalence definition. An implementation must use the equivalence expressed by this algorithm.
Furthermore, Unicode requires that certain formatting characters be ignored (skipped over) during string comparisons. The algorithm and tables used for case equivalence also arrange to ignore these characters. An implementations must ignore the characters that are ignored by this algorithm.
Note:
Case-sensitive HFSX volumes do
not ignore the Unicode ignorable characters.
Those characters are significant for the purposes of
name comparion on case-sensitive HFSX.
The HFS Plus case-insensitive string
comparison algorithm is defined by the FastUnicodeCompare
routine, shown below. This routine returns a value that
tells the caller how the strings are ordered relative to
each other: whether the first string is less than, equal to,
or greater than the second string. An HFS Plus implementation
may use this routine directly, or use another routine that
produces the same relative ordering.
Note:
The FastUnicodeCompare routine does not handle
composed Unicode characters since they are illegal in HFS
Plus strings. As described in
Canonical
Decomposition, all HFS Plus strings must be fully
decomposed, with composing characters in canonical order.
An HFS Plus volume may be contained within an HFS volume in a way that makes the volume look like an HFS volume to systems without HFS Plus support. This has a two important advantages:
The rest of this section describes how the HFS wrapper is laid out and how the HFS Plus volume is embedded within the wrapper.
This section does not describe the HFS Plus volume format; instead, it describes additions to the HFS volume format that allow an HFS Plus volume (or some other volume) to be embedded in an HFS volume. However, as all Mac OS volumes are formatted with an HFS wrapper, all implementations should be able to parse the wrapper to find the embedded HFS Plus volume.Note:
An HFS Plus volume is not required to have an HFS wrapper.
In that case, the volume will start at the beginning of
the disk, and the volume header will be at offset 1024 bytes.
However, Apple software currently initializes all HFS Plus
volumes with an HFS wrapper.
An HFS volume always contains a Master Directory Block (MDB), at offset 1024 bytes. The MDB is similar to an HFS Plus volume header. In order to support volumes embedded within an HFS volume, several unused fields of the MDB have been changed, and are now used to indicate the type, location, and size of the embedded volume.
What was formerly the drVCSize field (at
offset 0x7C) is now named drEmbedSigWord. This
two-byte field contains a unique value that identifies the
type of embedded volume. When an HFS Plus volume is
embedded, drEmbedSigWord must be
kHFSPlusSigWord ('H+'), the same
value stored in the signature field of an HFS
Plus volume header.
What were formerly the drVBMCSize and
drCtlCSize fields (at offset 0x7E)
have been combined into a single field occupying four bytes.
The new structure is named drEmbedExtent and is
of type HFSExtentDescriptor. It contains the
starting allocation block number (startBlock)
where the embedded volume begins and number of allocation
blocks (blockCount ) the embedded volume
occupies. The embedded volume must be contiguous. Both of
these values are in terms of the HFS wrapper's allocation
blocks, not HFS Plus allocation blocks.
Note:
The description of the HFS volume format in
Inside
Macintosh: Files describes these fields as being used to
store the size of various caches, and labels each one as
"used internally".
To actually find the embedded volume's location on disk,
an implementation must use the drAlBlkSiz and
drAlBlSt fields of the MDB. The
drAlBlkSiz field contains the size (in bytes)
of the HFS allocation blocks. The drAlBlSt
field contains the offset, in 512-byte blocks, of the
wrapper's allocation block 0 relative to the start of the
volume.
In order to prevent accidentally changing the files in
the HFS wrapper, the wrapper volume must be marked as
software-write-protected by setting
kHFSVolumeSoftwareLockBit in the
drAtrb (volume attributes) field of the MDB.
All correct HFS implementations will prevent any changes to
the wrapper volume.
To improve performance of HFS Plus volumes, the size of
the wrapper's allocation blocks should be a multiple of the
size of the HFS Plus volume's allocation blocks. In
addition, the wrapper's allocation block start
(drAlBlSt) should be a multiple of the HFS Plus
volume's allocation block size (or perhaps 4 KB, if the HFS
Plus allocation blocks are larger). If these recommendations
are followed, the HFS Plus allocation blocks will be
properly aligned on the disk. And, if the HFS Plus
allocation block size is a multiple of the sector size,
then blocking and deblocking at the device driver level
will be minimized.
The space occupied by the embedded volume must be marked as allocated in the HFS wrapper's volume bitmap (similar to the HFS Plus allocation file) and placed in the HFS wrapper's bad block file (similar to the HFS Plus bad block file). This doesn't mean the blocks are actually bad; it merely prevents the HFS Plus volume from being overwritten by newly created files in the HFS wrapper, being deleted accidentally, or being marked as free, usable space by HFS disk repair utilities.
The kHFSVolumeSparedBlocksMask bit of the
drAtrb (volume attributes) field of the MDB
must be set to indicate that the volume has a bad blocks
file.
As initialized by the Mac OS Disk Initialization Package, the HFS wrapper volume contains five files in the root folder.
In addition, the root folder is set as the blessed folder
by placing its folder ID in the first SInt32 of
the drFndrInfo (Finder information) field of
the MDB.
An HFS Plus volume is a complex data structure, consisting of many different inter-related data structures. Inconsistencies between these data structures could cause serious data loss. When an HFS Plus implementation mounts a volume, it must perform basic consistency checks to ensure that the volume is consistent. In addition, the implementation may choose to implement other, more advanced, consistency checks.
Many of these consistency checks take a significant amount of time to run. While a safe implementation might run these checks every time a volume is mounted, most implementations will want to rely on the correctness of the previous implementation that modified the disk. The implementation may avoid unnecessary checking by determining whether the volume was last unmounted cleanly. If it was, the implementation may choose to skip a consistency check.
An implementation can determine whether a volume was unmounted cleanly by looking at various flag bits in the volume header. See Volume Attributes for details.
For an HFS Plus volume to work correctly with
many implementations, it is vital that the nextCatalogID
field of the volume header be greater than
all CNIDs currently used in the
catalog file. The algorithm to ensure this is as follows.
nextCatalogID to a
value greater than it.Note:
The consistency check of nextCatalogID must be
skipped if kHFSCatalogNodeIDsReusedBit is set
in the attributes field of the
volume header.
For an HFS Plus volume to work correctly, it's vital that any allocation block in use by file system structures be marked as allocated in the allocation file. The algorithm to ensure this is as follows:
HFSPlusForkData structures for the data and
resource forks).Volume format specifications are fun exhausting.
Inside Macintosh: Files, especially the Organization on Volumes section.
Finder Interface Reference section of the Carbon user experience documentation. Technical Note 1189: The Monster Disk Driver Technote, especially the Secrets of the Partition Map section.Algorithms in C, Robert Sedgewick, Addison-Wesley, 1992, especially the section on B-trees.
Updated to clarify the allocation block usage and placement of the alternate volume header on volumes where the disk size is not an even multiple of the allocation block size.
2-Mar-2004
Added information about the journal.
Updated the HFS Plus Permissions section to describe the way Mac OS X uses permissions on disk.
Added the Hard Links and Symbolic Links sections to describe how Mac OS X implements hard and symbolic links.
Added information on how catalog node IDs can wrap around and be reused, in which case the CNID consistency check must be skipped.
Added details about the Finder information in the volume header. Added a section on the Finder information for files and directories.
Added a section about HFSX, an extension of HFS Plus that allows for case-sensitive file and directory names.
Added a section about how Mac OS X version 10.3 uses a Metadata Zone and adaptive hot file clustering.
|
|
含蓄的莲藕 · shell 每秒执行一次-掘金 2 年前 |