--- src/sbin/jscan/jscan.8 2005/08/28 05:13:53 1.5 +++ src/sbin/jscan/jscan.8 2005/09/06 06:42:44 1.6 @@ -41,28 +41,241 @@ .Nd journal file processing program .Sh SYNOPSIS .Nm -.Fl 2rd -.Op Ar journal_files +.Fl 2duF +.Op Fl c Ar count[k,m,g,t] +.Op Fl D Ar directory +.Op Fl m Ar mirror_transid_file/none +.Op Fl o/O Ar output_transid_file/none +.Op Fl s Ar size[k,m,g,t] +.Op Fl w/W Ar journal_prefix +.Op Ar journal_prefix/file .Pp .Sh DESCRIPTION The .Nm -utility scans journal files for the purposes of debugging dumps, restoration, -undo, mirroring, and other journaling features. +utility scans journal file or input stream for the purposes of debugging +dumps, restoration, undo, mirroring, and other journaling features. .Bl -tag -width indent .It Fl 2 Implement the full-duplex acknowledgement protocol on the input descriptor. Note that shell pipes are full-duplex and can be used with this option. +.It Fl c Ar count +Specify the number of transaction records which should be scanned, then exit. +This option is typically used along with +.Fl m +to limit the amount of work that +.Nm +does, giving you the ability to incrementally run a mirror forwards or +backwards. It is not usually used when piping in a live journal, but it +can be. .It Fl d -dumps the contents of the journaling file in a human readable format. -.It Fl r -will cause the journal be scanned backwards. -Transactions will be dumped in reverse order. -However, note that due to parallelism the order will not be the exact reverse -of a forward scan, since large transactions are not dumped until all stream -segments have been collected for them. +Display the contents of the journaling file or stream in a human readable +format on stderr. Note that stdout is used only for +.Fl o . +.It Fl D Ar directory +Specify the base directory for the mirroring option. +.It Fl m Ar mirror_transid_file/none +Generate a mirror in the directory specified by +.Fl D +or, if not specified, the current directory. +The +.Ar mirror_transid_file +will be used to track the transaction id representing the current +syncnronization point for the mirror. The keyword +.Ar none +may be specified if no tracking file is desired. However, if no tracking +file is specified it will not be possible to roll the mirror forwards or +backwards or restart the journaling stream being used to generate the mirror. +.Pp +It is important to note that journaling streams can contain meta-transactions +representing huge, multi-gigabyte operations. If the journaling data is +not being recorded to regular files via +.Fl w/W +it is possible that +.Nm +could run itself out of memory trying to record the meta-transactions. +In addition, the mirror would not be restartable. If the journaling data +is being recorded via +.Fl w/W +and a mirroring transaction id file is being kept, the mirror can be +restarted. +.Pp +While it is possible to run a journaling stream directly into a mirror, +it is more typical to file the jornaling stream with +.Fl w +and catch the mirror up as a batch job with the journaling file set prefix +specified as the input every so often. This way the system operator can +use other +.Nm +commands to, for example, run a mirror backwards and forwards in time. +.It Fl o/O Ar output_transid_file/none +Generate a journaling stream on stdout using the specified file to track +the transaction id to help with restarts. +The +.Fl o +option indicates a half-duplex output stream while the +.Fl O +option indicates a full-duplex (ACK protocol) output stream. +.Pp +This option is not really designed to output to regular files because it +does NOT necessarily weed out duplicate records. When both the input +stream and output stream are full-duplex and +.Fl w/W +is not specified, +.Nm +acts as a stateless transceiver and the input stream is not acked until +an ack is received from the output stream. +.Pp +This option is most typically used in conjuction with +.Fl w/W . +In this case the ACK protocol is handled independantly for the input side +and the output side uses the journaling data recorded by +.Fl w/W +as a buffer. +.Pp +In half-duplex output mode the output transaction id file is updated +after a raw transaction record has been successfully written to stdout. +In full-duplex output mode the file is only updated with ACK data returned +on the stdout descriptor. +.Pp +As with the +.Fl m +option, you can combine +.Fl o +in a journaling pipe with other options, but if you are trying to use it +as a buffer it may be better to have it separately pull its data off of +a journaling file set generated via +.Fl w . +.It Fl s Ar size +Change the size limit for rotating files created via +.Fl w . +The default is 100M. Values are in bytes or may be suffixed with k, +m, or g. +If a raw transaction causes the file's size limit to be exceeded, a new file +will be created. If a raw transaction is, in-whole, larger the the file's +size limit, the raw transaction will still be fully written to the file before +a new file is created. Raw transactions are typically limited to the size +of the source system's memory FIFO. This option is typically used to size +journaling files to fit onto the appropriate backup media or to provide +bite-sized chunks for other programs to injest. +.Pp +When restarting a journal, a new sequence number will always be chosen for +the resumption of data recording. No existing file will be appended to when +.Nm +is reinvoked. +.It Fl u +Will cause the journal to be scanned backwards (requires seekable media). +Transactions will be dumped in reverse order. If mirroring, the UNDO +data will be executed. If not specified, 1 hour's worth of data will be +undone. Can only be used with a journaling file or journaling prefix +as the input. +.It Fl w Ar prefix +The received journaling stream is recorded in journaling files named +.Ar . +and the current transaction id is tracked in a file named +.Ar .transid . +A journaling file is closed out and a new file with the next sequence +number is created once the file surpasses 100MB. +.Pp +This option is robust across restarts. The current transaction id +will be read and the input stream will be skipped until it is reached. +If the input is a journaling file or prefix set, +.Nm +will be able to quickly seek to the restart point. +.Pp +NOTE: If +you are generating a mirror with the same command via +.Fl m , +and the journaling data input is a stream rather then a file or prefix +set, you must use +.Fl w/W +if you want the mirror to be restartable. This is because while we can +pick up the transaction id where we left off, that raw transaction id may +have cut a larger meta-transaction in half and the mirroring code will +not be able to access the whole of the transaction unless it has a file +or prefix set to work with. +.It Fl W Ar prefix +Similar to +.Fl w +except that the journaling files created are strictly temporary and will +be deleted once they exceed the size limit AND the related meta-transactions +have been completed. +.Pp +If combined with +.Fl m , +the meta-transactions are considered to be completed only when the mirror +finishes executing them. It is possible for several sequence number files +to build if a particularly large meta-transaction is coming down the pipe. +.Pp +If combined with +.Fl o/O , +the meta-transactions are considered to be completed when the data has +been successfully written out to the pipe in half duplex mode, or when +the ACK has been received in full-duplex mode. +.Pp +If both +.Fl m +and +.Fl w/W +is used, the journaling data files are only deleted when both actions +no longer need the data. +.It Fl F +Forces +.Nm +to fsync() after updating a journaling file prior to acknowledging the +data or updating a transaction-id-tracking file. If specified twice, +.Nm +will also fsync() after updating the transaction-id-tracking file. +.It Ar journal_prefix/file +Specify the input to jscan. This can be a journaling file set prefix +or it can be a plain file. If no input file is specified, stdin is +assumed. Note that when generating a mirror from a stdin stream, the +mirror will not be restartable unless +.Fl w/W +is also used. +.Pp .El .Pp +.Sh OPERATIONAL NOTES +It is often important to be able to quickly stage journaled data through +a dedicated backup machine on a LAN. There are several places where data +can be buffered and staged out. +.Pp +The machine generating the journal typically buffers several megabytes of +journal data in the kernel. This local machine can pipe that data to +.Nm +or some other locally run program to add another buffering stage, or you +can directly attach a TCP connection to the kernel's journaling output. +.Pp +The LAN backup box typically buffers gigabytes worth of data by running +multiple jscan's. The jscan on the receiving end of the TCP or pipe (for +example, via ssh) typically records the data via the +.Fl w +option, and then runs other +.Nm +programs from scripts or cron to take that data and copy it to your +off-site backup machine. Other jscan programs may use the same data +set to generate mirrors or other backup streams. +.Pp +It should be noted that if +.Fl w/W +is specified, both mirroring mode and output mode will internally +fork the program once the appropriate synchronization point has been reached, +effectively decoupling their operation, and read all of their data via +the journaling files written out by the master program. In particular, +blockages in the mirroring and output code will not effect our ability +to buffer the journaling input data via +.Fl w/W . +If +.Fl w/W +is not specified then neither the mirroring or output modes will fork. Under +these conditions, if the input is a stream rather then a file +.Nm +will be forced to buffer meta-transactions (for mirroring) entirely in +memory, which could present a serious problem since a single meta-transaction +can exceed a gigabyte (e.g. if someone were to do a single write() system +call writing a gigabyte all in one go). +.Pp .Sh SEE ALSO .Xr mountctl 8 .Sh CAVEATS