Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 3441548066 | |||
| 6074f101ff |
@@ -1,7 +1,7 @@
|
||||
# darktable-sync — Projektdokumentation für Claude
|
||||
|
||||
## Zweck
|
||||
Bash-Skripte zur bidirektionalen Synchronisation der Darktable-Datenbank und Fotobibliothek zwischen lokalem Rechner und einem Server via rsync/SSH. Entwickelt für Linux mit systemd.
|
||||
Bash-Skripte zur bidirektionalen Synchronisation der Darktable-Datenbank und Fotobibliothek zwischen lokalem Rechner und einem Server via Unison/SSH. Entwickelt für Linux mit systemd.
|
||||
|
||||
## Architektur
|
||||
|
||||
@@ -12,42 +12,45 @@ darktable_common.sh → Gemeinsame Hilfsfunktionen (wird per source eingebund
|
||||
```
|
||||
|
||||
### Ablauf darktable_sync.sh
|
||||
1. Dependencies prüfen (rsync, ssh, notify-send, darktable)
|
||||
1. Dependencies prüfen (unison, ssh, notify-send, darktable)
|
||||
2. Config laden und validieren
|
||||
3. Lock erwerben (atomares `mkdir`)
|
||||
4. Dry-Run-Modus prüfen (Standard: Trockenlauf; `--execute`/`-e` für echten Sync)
|
||||
5. Darktable-Prozess prüfen (Sync verboten wenn darktable läuft)
|
||||
6. Server-Erreichbarkeit prüfen → `sync_pending` bei Fehler
|
||||
7. Active-Marker prüfen (verhindert gleichzeitige Clients)
|
||||
8. Darktable-Versionen abgleichen (Major.Minor müssen übereinstimmen)
|
||||
9. Sync-Token prüfen (Konflikterkennung bei mehreren Clients)
|
||||
10. DB-Backup erstellen (library.db.bak, data.db.bak) — nur bei `--execute`
|
||||
11. Backup-Verzeichnisse anlegen (`${LOCAL_PHOTO_DIR}-bak`, `${LOCAL_DARKTABLE_DB_DIR}-bak`)
|
||||
12. Upload: DB und Fotos mit `--delete` (lokal gelöscht → Server gelöscht)
|
||||
13. Download: DB und Fotos mit `--delete --backup --backup-dir` (lokal gelöscht → ins `-bak`-Verzeichnis)
|
||||
14. Sync-Token und Versionsdatei speichern
|
||||
15. Alte Backups bereinigen (`cleanup_old_backups`, >730 Tage)
|
||||
7. Unison-Versionen abgleichen (exakt gleiche Version auf beiden Seiten erforderlich)
|
||||
8. Active-Marker prüfen (verhindert gleichzeitige Clients)
|
||||
9. Darktable-Versionen abgleichen (Major.Minor müssen übereinstimmen)
|
||||
10. Sync-Token prüfen (Konflikterkennung bei mehreren Clients) → `-force local`/`-force remote`/`-prefer newer`
|
||||
11. DB-Backup erstellen (library.db.bak, data.db.bak) — nur bei `--execute`
|
||||
12. Backup-Verzeichnisse anlegen (`${LOCAL_PHOTO_DIR}-bak`, `${LOCAL_DARKTABLE_DB_DIR}-bak`)
|
||||
13. DB-Sync: bidirektional via Unison (lokal gelöscht + im Archiv → Server löschen; nie lokal vorhanden → herunterladen)
|
||||
14. Foto-Sync: bidirektional via Unison mit `-prefer newer`
|
||||
15. Sync-Token und Versionsdatei speichern
|
||||
16. Alte Backups bereinigen (`cleanup_old_backups`, >730 Tage)
|
||||
|
||||
## Konfiguration
|
||||
- Datei: `~/.config/darktable-sync/.env` (Permissions: 600)
|
||||
- Wichtige Variablen: `SERVER_IP`, `SERVER_USER`, `SERVER_SSH_PORT`, `SERVER_DB_DIR`, `SERVER_PHOTO_DIR`, `LOCAL_DARKTABLE_DB_DIR`, `LOCAL_PHOTO_DIR`, `DARKTABLE_BIN`, `SYNC_BIN`
|
||||
- Vorlage: `.env.example`
|
||||
|
||||
## rsync-Flags
|
||||
## Unison-Flags
|
||||
| Operation | Flags |
|
||||
|---|---|
|
||||
| Upload DB | `-uavh --itemize-changes --delete --exclude '*.lock' --exclude 'darktable_version'` |
|
||||
| Upload Fotos | `-uavh --itemize-changes --delete --exclude '*.lock'` |
|
||||
| Download DB | `-uavh --itemize-changes --delete --backup --backup-dir="${LOCAL_DARKTABLE_DB_DIR}-bak"` |
|
||||
| Download Fotos | `-uavh --itemize-changes --delete --backup --backup-dir="${LOCAL_PHOTO_DIR}-bak"` |
|
||||
| DB-Sync | `-batch -times -auto -prefer newer -ignore "Name *.lock" -ignore "Name darktable_version" -backup "Name *" -backupdir "${LOCAL_DARKTABLE_DB_DIR}-bak"` |
|
||||
| Foto-Sync | `-batch -times -auto -prefer newer -ignore "Name *.lock" -backup "Name *" -backupdir "${LOCAL_PHOTO_DIR}-bak"` |
|
||||
| Bei DB-Konflikt (upload) | `-force local` statt `-prefer newer` |
|
||||
| Bei DB-Konflikt (download) | `-force remote` für DB und Fotos |
|
||||
|
||||
Dry-Run: `RSYNC_DRY_FLAG=(--dry-run)` wird zu allen Aufrufen hinzugefügt.
|
||||
Dry-Run: `UNISON_DRY_FLAG=(-dryrun)` wird zu allen Aufrufen hinzugefügt.
|
||||
|
||||
## itemize-changes Format
|
||||
- `<f+++++++++` — neue Datei (Upload, an Remote gesendet)
|
||||
- `>f+++++++++` — neue Datei (Download, lokal empfangen)
|
||||
- `<f` / `>f` ohne `+` — aktualisiert
|
||||
- `*deleting` — gelöscht
|
||||
## Unison-Output-Format (Dryrun)
|
||||
- `new file ----> path` — neue Datei (Upload, an Server)
|
||||
- `<---- new file path` — neue Datei (Download, lokal)
|
||||
- `changed ----> path` — aktualisiert (Upload)
|
||||
- `<---- changed path` — aktualisiert (Download)
|
||||
- `deleted ----> path` — gelöscht (Upload)
|
||||
- `<---- deleted path` — gelöscht (Download)
|
||||
|
||||
## Sicherheitsmechanismen
|
||||
- `validate_path`: Blockiert `' " ; | & \` $ ( ) \` und `..` in Pfaden (Server- UND lokale Pfade)
|
||||
@@ -58,9 +61,9 @@ Dry-Run: `RSYNC_DRY_FLAG=(--dry-run)` wird zu allen Aufrufen hinzugefügt.
|
||||
|
||||
## Test-Infrastruktur
|
||||
- Framework: **BATS** (`tests/*.bats`)
|
||||
- Stubs in `tests/stubs/`: `rsync`, `ssh`, `darktable`, `notify-send`, `pgrep`, `zenity`, `kdialog`
|
||||
- Stubs in `tests/stubs/`: `unison`, `ssh`, `darktable`, `notify-send`, `pgrep`, `zenity`, `kdialog`
|
||||
- Stub-Aktivierung: `run_with_stubs` setzt `tests/stubs/` vorne in PATH
|
||||
- Steuerung via Env-Variablen: `RSYNC_STUB_FAIL`, `RSYNC_STUB_DRY_LINES`, `RSYNC_STUB_ARGS_FILE`, `SSH_STUB_FAIL`, `SSH_STUB_OUTPUT`, `PGREP_STUB_FOUND`
|
||||
- Steuerung via Env-Variablen: `UNISON_STUB_FAIL`, `UNISON_STUB_DRY_LINES`, `UNISON_STUB_ARGS_FILE`, `SSH_STUB_FAIL`, `SSH_STUB_OUTPUT`, `UNISON_SERVER_VERSION`, `PGREP_STUB_FOUND`
|
||||
- Setup: `create_valid_env` in `tests/helpers/setup.bash`; jeder Test bekommt isoliertes `$HOME` in `$BATS_TMPDIR`
|
||||
- Tests ausführen: `bats tests/`
|
||||
|
||||
|
||||
+8
-2
@@ -88,11 +88,17 @@ echo ""
|
||||
|
||||
echo "Abhaengigkeiten pruefen..."
|
||||
|
||||
REQUIRED_CMDS=("rsync" "notify-send" "darktable" "systemctl" "ssh")
|
||||
REQUIRED_CMDS=("unison" "notify-send" "darktable" "systemctl" "ssh")
|
||||
for cmd in "${REQUIRED_CMDS[@]}"; do
|
||||
if ! command -v "$cmd" >/dev/null 2>&1; then
|
||||
echo "Fehler: '$cmd' ist nicht installiert."
|
||||
echo " Installieren mit: sudo apt install $cmd"
|
||||
if [ "$cmd" = "unison" ]; then
|
||||
echo " Installieren mit: sudo apt install unison"
|
||||
echo " WICHTIG: Unison muss auch auf dem Server installiert sein (gleiche Version):"
|
||||
echo " ssh $SERVER_USER@$SERVER_IP sudo apt install unison"
|
||||
else
|
||||
echo " Installieren mit: sudo apt install $cmd"
|
||||
fi
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
@@ -91,6 +91,44 @@ log_error() {
|
||||
echo "FEHLER: $*" >&2
|
||||
}
|
||||
|
||||
# Prüft ob lokale und Server-Unison-Version übereinstimmen.
|
||||
# Unison erfordert exakt gleiche Version auf beiden Seiten.
|
||||
check_unison_versions() {
|
||||
local local_ver server_ver
|
||||
local_ver=$(unison -version 2>/dev/null | head -1 || true)
|
||||
server_ver=$(ssh_server "unison -version 2>/dev/null | head -1 || echo 'nicht installiert'" || true)
|
||||
log " Unison lokal: ${local_ver:-unbekannt}"
|
||||
log " Unison Server: ${server_ver:-unbekannt}"
|
||||
if [ "${local_ver:-}" != "${server_ver:-}" ]; then
|
||||
log_error "Unison-Version stimmt nicht überein: lokal='$local_ver', server='$server_ver'"
|
||||
log_error "Gleiche Unison-Version auf beiden Seiten installieren: sudo apt install unison"
|
||||
notify-send "Darktable Sync – Fehler" \
|
||||
"Unison-Versionsmismatch\nLokal: $local_ver\nServer: $server_ver" -u critical
|
||||
touch "$CONFIG_DIR/sync_pending"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Zählt Änderungen aus Unison-Dryrun-Output nach Richtung und Typ.
|
||||
count_unison_changes() {
|
||||
local log_file="$1" direction="$2" type="$3"
|
||||
case "$direction:$type" in
|
||||
up:new) grep -cE 'new file.*---->' "$log_file" 2>/dev/null ;;
|
||||
up:changed) grep -cE 'changed.*---->' "$log_file" 2>/dev/null ;;
|
||||
up:deleted) grep -cE 'deleted.*---->' "$log_file" 2>/dev/null ;;
|
||||
down:new) grep -cE '<----.*new file' "$log_file" 2>/dev/null ;;
|
||||
down:changed) grep -cE '<----.*changed' "$log_file" 2>/dev/null ;;
|
||||
down:deleted) grep -cE '<----.*deleted' "$log_file" 2>/dev/null ;;
|
||||
*) echo 0; return ;;
|
||||
esac || echo 0
|
||||
}
|
||||
|
||||
# Liest die Anzahl übertragener Dateien aus dem Unison-Execute-Output.
|
||||
count_unison_transferred() {
|
||||
local log_file="$1"
|
||||
grep -oP '\d+(?= item\(s\) transferred)' "$log_file" 2>/dev/null | tail -1 || echo 0
|
||||
}
|
||||
|
||||
classify_filetype() {
|
||||
local file="$1"
|
||||
local ext="${file##*.}"; ext="${ext,,}"
|
||||
@@ -103,15 +141,27 @@ classify_filetype() {
|
||||
esac
|
||||
}
|
||||
|
||||
format_rsync_details() {
|
||||
format_unison_details() {
|
||||
local log_file="$1" direction_label="$2" direction="$3"
|
||||
[ -f "$log_file" ] || return 0
|
||||
local prefix; [ "$direction" = "up" ] && prefix="<f" || prefix=">f"
|
||||
|
||||
local neu_pat upd_pat del_pat
|
||||
if [ "$direction" = "up" ]; then
|
||||
neu_pat='new file.*---->'
|
||||
upd_pat='changed.*---->'
|
||||
del_pat='deleted.*---->'
|
||||
else
|
||||
neu_pat='<----.*new file'
|
||||
upd_pat='<----.*changed'
|
||||
del_pat='<----.*deleted'
|
||||
fi
|
||||
|
||||
local label pattern files
|
||||
while IFS=: read -r label pattern; do
|
||||
for entry in "neu:$neu_pat" "aktualisiert:$upd_pat" "gelöscht:$del_pat"; do
|
||||
label="${entry%%:*}"
|
||||
pattern="${entry#*:}"
|
||||
files=$(grep -E "$pattern" "$log_file" 2>/dev/null \
|
||||
| sed 's/^[^ ]* *//' | sort) || true
|
||||
| awk '{print $NF}' | sort) || true
|
||||
[ -n "$files" ] || continue
|
||||
|
||||
declare -A dir_foto=() dir_xmp=() dir_sonstige=()
|
||||
@@ -151,11 +201,7 @@ format_rsync_details() {
|
||||
fi
|
||||
|
||||
unset dir_foto dir_xmp dir_sonstige
|
||||
done <<EOF
|
||||
neu:^${prefix}[+]{9}
|
||||
aktualisiert:^${prefix}[^+]
|
||||
gelöscht:^\*deleting
|
||||
EOF
|
||||
done
|
||||
}
|
||||
|
||||
cleanup_old_backups() {
|
||||
|
||||
+79
-121
@@ -7,7 +7,7 @@ source "$SCRIPT_DIR/darktable_common.sh"
|
||||
|
||||
log_step "Darktable Sync gestartet (PID $$, Argumente: ${*:-keine})"
|
||||
|
||||
check_dependency rsync
|
||||
check_dependency unison
|
||||
check_dependency ssh openssh-client
|
||||
check_dependency notify-send libnotify-bin
|
||||
check_dependency darktable
|
||||
@@ -32,15 +32,6 @@ for arg in "$@"; do
|
||||
esac
|
||||
done
|
||||
|
||||
count_synced_files() {
|
||||
local log_file="$1" direction="$2" count=0
|
||||
case "$direction" in
|
||||
up) count=$(grep -cE '^<f|^cd' "$log_file" 2>/dev/null) || count=0 ;;
|
||||
down) count=$(grep -cE '^>f|^cd' "$log_file" 2>/dev/null) || count=0 ;;
|
||||
esac
|
||||
echo "$count"
|
||||
}
|
||||
|
||||
LOCKDIR="$CONFIG_DIR/sync.lock"
|
||||
LOCKPID="$LOCKDIR/pid"
|
||||
TMPFILES=()
|
||||
@@ -101,6 +92,10 @@ if [ "$SHOW_NOTIFY_START_STOP" = true ]; then
|
||||
notify-send "Darktable Sync" "Sync gestartet..." -t 3000
|
||||
fi
|
||||
|
||||
log "Unison-Versionen prüfen..."
|
||||
check_unison_versions
|
||||
log "Unison-Versionen übereinstimmend."
|
||||
|
||||
log "Active-Marker auf Server prüfen..."
|
||||
ACTIVE=$(ssh_server "cat '$SERVER_DB_DIR/darktable.active' 2>/dev/null || true")
|
||||
if [ -n "$ACTIVE" ]; then
|
||||
@@ -138,7 +133,9 @@ SERVER_TOKEN=$(server_db_mtime)
|
||||
log " Gespeicherter Token: ${SAVED_TOKEN:-keiner (erster Sync)}"
|
||||
log " Aktueller Server-Token: $SERVER_TOKEN"
|
||||
|
||||
UPLOAD_ALLOWED=true
|
||||
UNISON_DB_FLAGS=(-prefer newer)
|
||||
UNISON_PHOTO_FLAGS=(-prefer newer)
|
||||
|
||||
if [ -n "$SAVED_TOKEN" ] && [ "$SAVED_TOKEN" != "$SERVER_TOKEN" ]; then
|
||||
log "WARNUNG: Token-Konflikt (gespeichert=$SAVED_TOKEN, server=$SERVER_TOKEN) – Benutzer wird gefragt."
|
||||
RESOLUTION=$(ask_conflict_resolution)
|
||||
@@ -146,18 +143,20 @@ if [ -n "$SAVED_TOKEN" ] && [ "$SAVED_TOKEN" != "$SERVER_TOKEN" ]; then
|
||||
case "$RESOLUTION" in
|
||||
upload)
|
||||
log "Upload erzwungen – lokale Version überschreibt Server."
|
||||
UNISON_DB_FLAGS=(-force local)
|
||||
;;
|
||||
abort)
|
||||
log "Sync abgebrochen durch Benutzer."
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
log "Nur Download – Server-Stand wird übernommen."
|
||||
UPLOAD_ALLOWED=false
|
||||
log "Download – Server-Stand wird übernommen."
|
||||
UNISON_DB_FLAGS=(-force remote)
|
||||
UNISON_PHOTO_FLAGS=(-force remote)
|
||||
;;
|
||||
esac
|
||||
else
|
||||
log "Token stimmt überein – Upload erlaubt."
|
||||
log "Token stimmt überein – bidirektionaler Sync mit neuerer Version bevorzugt."
|
||||
fi
|
||||
|
||||
if [ "$DRY_RUN" = false ]; then
|
||||
@@ -171,14 +170,10 @@ fi
|
||||
|
||||
SYNC_LOG=$(mktemp)
|
||||
TMPFILES+=("$SYNC_LOG")
|
||||
UPLOAD_LOG_DB=$(mktemp)
|
||||
TMPFILES+=("$UPLOAD_LOG_DB")
|
||||
UPLOAD_LOG_PHOTOS=$(mktemp)
|
||||
TMPFILES+=("$UPLOAD_LOG_PHOTOS")
|
||||
DOWNLOAD_LOG_DB=$(mktemp)
|
||||
TMPFILES+=("$DOWNLOAD_LOG_DB")
|
||||
DOWNLOAD_LOG_PHOTOS=$(mktemp)
|
||||
TMPFILES+=("$DOWNLOAD_LOG_PHOTOS")
|
||||
UNISON_LOG_DB=$(mktemp)
|
||||
TMPFILES+=("$UNISON_LOG_DB")
|
||||
UNISON_LOG_PHOTOS=$(mktemp)
|
||||
TMPFILES+=("$UNISON_LOG_PHOTOS")
|
||||
|
||||
BACKUP_PHOTO_DIR="${LOCAL_PHOTO_DIR}-bak"
|
||||
BACKUP_DB_DIR="${LOCAL_DARKTABLE_DB_DIR}-bak"
|
||||
@@ -187,95 +182,61 @@ if [ "$DRY_RUN" = false ]; then
|
||||
mkdir -p "$BACKUP_PHOTO_DIR" "$BACKUP_DB_DIR"
|
||||
fi
|
||||
|
||||
RSYNC_DRY_FLAG=()
|
||||
[ "$DRY_RUN" = true ] && RSYNC_DRY_FLAG=(--dry-run)
|
||||
UNISON_DRY_FLAG=()
|
||||
[ "$DRY_RUN" = true ] && UNISON_DRY_FLAG=(-dryrun)
|
||||
[ "$DRY_RUN" = true ] && DRY_SUFFIX=" (Trockenlauf)" || DRY_SUFFIX=""
|
||||
|
||||
SENT_DB=0
|
||||
SENT_PHOTOS=0
|
||||
|
||||
if [ "$UPLOAD_ALLOWED" = true ]; then
|
||||
log_step "Upload: Datenbank${DRY_SUFFIX}"
|
||||
log " Quelle: $LOCAL_DARKTABLE_DB_DIR/"
|
||||
log " Ziel: $SERVER_USER@$SERVER_IP:$SERVER_DB_DIR/"
|
||||
if [ -z "$(ls -A "$LOCAL_DARKTABLE_DB_DIR" 2>/dev/null)" ]; then
|
||||
log_error "Upload abgebrochen: Quellverzeichnis leer ($LOCAL_DARKTABLE_DB_DIR). Falscher Mount?"
|
||||
touch "$CONFIG_DIR/sync_pending"
|
||||
exit 1
|
||||
fi
|
||||
if ! rsync -uavh --itemize-changes --delete \
|
||||
"${RSYNC_DRY_FLAG[@]}" \
|
||||
--exclude '*.lock' \
|
||||
--exclude 'darktable_version' \
|
||||
-e "ssh -p $SERVER_SSH_PORT" \
|
||||
"$LOCAL_DARKTABLE_DB_DIR/" "$SERVER_USER@$SERVER_IP:$SERVER_DB_DIR/" \
|
||||
2>&1 | tee -a "$SYNC_LOG" "$UPLOAD_LOG_DB"; then
|
||||
log_error "Upload Datenbank fehlgeschlagen (Quelle: $LOCAL_DARKTABLE_DB_DIR)"
|
||||
touch "$CONFIG_DIR/sync_pending"
|
||||
exit 1
|
||||
fi
|
||||
SENT_DB=$(count_synced_files "$UPLOAD_LOG_DB" "up")
|
||||
log "Datenbank-Upload abgeschlossen: $SENT_DB Datei(en) übertragen."
|
||||
|
||||
log_step "Upload: Fotos${DRY_SUFFIX}"
|
||||
log " Quelle: $LOCAL_PHOTO_DIR/"
|
||||
log " Ziel: $SERVER_USER@$SERVER_IP:$SERVER_PHOTO_DIR/"
|
||||
if [ -z "$(ls -A "$LOCAL_PHOTO_DIR" 2>/dev/null)" ]; then
|
||||
log_error "Upload abgebrochen: Quellverzeichnis leer ($LOCAL_PHOTO_DIR). Falscher Mount?"
|
||||
touch "$CONFIG_DIR/sync_pending"
|
||||
exit 1
|
||||
fi
|
||||
if ! rsync -uavh --itemize-changes --delete \
|
||||
"${RSYNC_DRY_FLAG[@]}" \
|
||||
--exclude '*.lock' \
|
||||
-e "ssh -p $SERVER_SSH_PORT" \
|
||||
"$LOCAL_PHOTO_DIR/" "$SERVER_USER@$SERVER_IP:$SERVER_PHOTO_DIR/" \
|
||||
2>&1 | tee -a "$SYNC_LOG" "$UPLOAD_LOG_PHOTOS"; then
|
||||
log_error "Upload Fotos fehlgeschlagen (Quelle: $LOCAL_PHOTO_DIR)"
|
||||
touch "$CONFIG_DIR/sync_pending"
|
||||
exit 1
|
||||
fi
|
||||
SENT_PHOTOS=$(count_synced_files "$UPLOAD_LOG_PHOTOS" "up")
|
||||
log "Foto-Upload abgeschlossen: $SENT_PHOTOS Datei(en) übertragen."
|
||||
else
|
||||
log "Upload übersprungen (Token-Konflikt)."
|
||||
fi
|
||||
|
||||
log_step "Download: Datenbank${DRY_SUFFIX}"
|
||||
log " Quelle: $SERVER_USER@$SERVER_IP:$SERVER_DB_DIR/"
|
||||
log " Ziel: $LOCAL_DARKTABLE_DB_DIR/"
|
||||
log " Backup: $BACKUP_DB_DIR/"
|
||||
if ! rsync -uavh --itemize-changes --delete \
|
||||
--backup --backup-dir="$BACKUP_DB_DIR" \
|
||||
"${RSYNC_DRY_FLAG[@]}" \
|
||||
--exclude '*.lock' \
|
||||
-e "ssh -p $SERVER_SSH_PORT" \
|
||||
"$SERVER_USER@$SERVER_IP:$SERVER_DB_DIR/" "$LOCAL_DARKTABLE_DB_DIR/" \
|
||||
2>&1 | tee -a "$SYNC_LOG" "$DOWNLOAD_LOG_DB"; then
|
||||
log_error "Download Datenbank fehlgeschlagen (Ziel: $LOCAL_DARKTABLE_DB_DIR)"
|
||||
if [ -z "$(ls -A "$LOCAL_DARKTABLE_DB_DIR" 2>/dev/null)" ]; then
|
||||
log_error "Sync abgebrochen: Quellverzeichnis leer ($LOCAL_DARKTABLE_DB_DIR). Falscher Mount?"
|
||||
touch "$CONFIG_DIR/sync_pending"
|
||||
exit 1
|
||||
fi
|
||||
RECEIVED_DB=$(count_synced_files "$DOWNLOAD_LOG_DB" "down")
|
||||
log "Datenbank-Download abgeschlossen: $RECEIVED_DB Datei(en) empfangen."
|
||||
|
||||
log_step "Download: Fotos${DRY_SUFFIX}"
|
||||
log " Quelle: $SERVER_USER@$SERVER_IP:$SERVER_PHOTO_DIR/"
|
||||
log " Ziel: $LOCAL_PHOTO_DIR/"
|
||||
log " Backup: $BACKUP_PHOTO_DIR/"
|
||||
if ! rsync -uavh --itemize-changes --delete \
|
||||
--backup --backup-dir="$BACKUP_PHOTO_DIR" \
|
||||
"${RSYNC_DRY_FLAG[@]}" \
|
||||
--exclude '*.lock' \
|
||||
-e "ssh -p $SERVER_SSH_PORT" \
|
||||
"$SERVER_USER@$SERVER_IP:$SERVER_PHOTO_DIR/" "$LOCAL_PHOTO_DIR/" \
|
||||
2>&1 | tee -a "$SYNC_LOG" "$DOWNLOAD_LOG_PHOTOS"; then
|
||||
log_error "Download Fotos fehlgeschlagen (Ziel: $LOCAL_PHOTO_DIR)"
|
||||
if [ -z "$(ls -A "$LOCAL_PHOTO_DIR" 2>/dev/null)" ]; then
|
||||
log_error "Sync abgebrochen: Quellverzeichnis leer ($LOCAL_PHOTO_DIR). Falscher Mount?"
|
||||
touch "$CONFIG_DIR/sync_pending"
|
||||
exit 1
|
||||
fi
|
||||
RECEIVED_PHOTOS=$(count_synced_files "$DOWNLOAD_LOG_PHOTOS" "down")
|
||||
log "Foto-Download abgeschlossen: $RECEIVED_PHOTOS Datei(en) empfangen."
|
||||
|
||||
log_step "Sync: Datenbank${DRY_SUFFIX}"
|
||||
log " Lokal: $LOCAL_DARKTABLE_DB_DIR/"
|
||||
log " Server: $SERVER_USER@$SERVER_IP:$SERVER_DB_DIR/"
|
||||
if ! unison "$LOCAL_DARKTABLE_DB_DIR" \
|
||||
"ssh://$SERVER_USER@$SERVER_IP/$SERVER_DB_DIR" \
|
||||
-batch -times -auto \
|
||||
"${UNISON_DB_FLAGS[@]}" \
|
||||
"${UNISON_DRY_FLAG[@]}" \
|
||||
-sshargs "-p $SERVER_SSH_PORT" \
|
||||
-ignore "Name *.lock" \
|
||||
-ignore "Name darktable_version" \
|
||||
-backup "Name *" \
|
||||
-backupdir "$BACKUP_DB_DIR" \
|
||||
2>&1 | tee -a "$SYNC_LOG" "$UNISON_LOG_DB"; then
|
||||
log_error "Sync Datenbank fehlgeschlagen ($LOCAL_DARKTABLE_DB_DIR)"
|
||||
touch "$CONFIG_DIR/sync_pending"
|
||||
exit 1
|
||||
fi
|
||||
CHANGED_DB=$(count_unison_transferred "$UNISON_LOG_DB")
|
||||
log "Datenbank-Sync abgeschlossen: $CHANGED_DB Datei(en) geändert."
|
||||
|
||||
log_step "Sync: Fotos${DRY_SUFFIX}"
|
||||
log " Lokal: $LOCAL_PHOTO_DIR/"
|
||||
log " Server: $SERVER_USER@$SERVER_IP:$SERVER_PHOTO_DIR/"
|
||||
if ! unison "$LOCAL_PHOTO_DIR" \
|
||||
"ssh://$SERVER_USER@$SERVER_IP/$SERVER_PHOTO_DIR" \
|
||||
-batch -times -auto \
|
||||
"${UNISON_PHOTO_FLAGS[@]}" \
|
||||
"${UNISON_DRY_FLAG[@]}" \
|
||||
-sshargs "-p $SERVER_SSH_PORT" \
|
||||
-ignore "Name *.lock" \
|
||||
-backup "Name *" \
|
||||
-backupdir "$BACKUP_PHOTO_DIR" \
|
||||
2>&1 | tee -a "$SYNC_LOG" "$UNISON_LOG_PHOTOS"; then
|
||||
log_error "Sync Fotos fehlgeschlagen ($LOCAL_PHOTO_DIR)"
|
||||
touch "$CONFIG_DIR/sync_pending"
|
||||
exit 1
|
||||
fi
|
||||
CHANGED_PHOTOS=$(count_unison_transferred "$UNISON_LOG_PHOTOS")
|
||||
log "Foto-Sync abgeschlossen: $CHANGED_PHOTOS Datei(en) geändert."
|
||||
|
||||
if [ "$DRY_RUN" = false ]; then
|
||||
NEW_TOKEN=$(server_db_mtime)
|
||||
@@ -293,19 +254,17 @@ if [ "$DRY_RUN" = false ]; then
|
||||
cleanup_old_backups "$BACKUP_DB_DIR"
|
||||
fi
|
||||
|
||||
TOTAL_SENT=$((SENT_DB + SENT_PHOTOS))
|
||||
TOTAL_RECEIVED=$((RECEIVED_DB + RECEIVED_PHOTOS))
|
||||
TOTAL_CHANGED=$((CHANGED_DB + CHANGED_PHOTOS))
|
||||
|
||||
if [ "$DRY_RUN" = true ]; then
|
||||
upload_log=$(cat "$UPLOAD_LOG_DB" "$UPLOAD_LOG_PHOTOS" 2>/dev/null || true)
|
||||
download_log=$(cat "$DOWNLOAD_LOG_DB" "$DOWNLOAD_LOG_PHOTOS" 2>/dev/null || true)
|
||||
all_log=$(cat "$UNISON_LOG_DB" "$UNISON_LOG_PHOTOS" 2>/dev/null || true)
|
||||
|
||||
UP_NEW=$( echo "$upload_log" | grep -cE '^<f[+]{9}' || true)
|
||||
UP_UPD=$( echo "$upload_log" | grep -E '^<f' | grep -cvE '^<f[+]{9}' || true)
|
||||
UP_DEL=$( echo "$upload_log" | grep -cE '^\*deleting' || true)
|
||||
DN_NEW=$( echo "$download_log" | grep -cE '^>f[+]{9}' || true)
|
||||
DN_UPD=$( echo "$download_log" | grep -E '^>f' | grep -cvE '^>f[+]{9}' || true)
|
||||
DN_DEL=$( echo "$download_log" | grep -cE '^\*deleting' || true)
|
||||
UP_NEW=$( echo "$all_log" | grep -cE 'new file.*---->' || true)
|
||||
UP_UPD=$( echo "$all_log" | grep -cE 'changed.*---->' || true)
|
||||
UP_DEL=$( echo "$all_log" | grep -cE 'deleted.*---->' || true)
|
||||
DN_NEW=$( echo "$all_log" | grep -cE '<----.*new file' || true)
|
||||
DN_UPD=$( echo "$all_log" | grep -cE '<----.*changed' || true)
|
||||
DN_DEL=$( echo "$all_log" | grep -cE '<----.*deleted' || true)
|
||||
|
||||
log_step "Trockenlauf-Ergebnis"
|
||||
log " Upload: $UP_NEW neu | $UP_UPD aktualisiert | $UP_DEL gelöscht"
|
||||
@@ -313,22 +272,21 @@ if [ "$DRY_RUN" = true ]; then
|
||||
|
||||
if [ "$((UP_NEW + UP_UPD + UP_DEL + DN_NEW + DN_UPD + DN_DEL))" -gt 0 ]; then
|
||||
if ask_user "Details" "Details der zu übertragenden Dateien anzeigen?"; then
|
||||
format_rsync_details "$UPLOAD_LOG_DB" "Upload" "up"
|
||||
format_rsync_details "$UPLOAD_LOG_PHOTOS" "Upload" "up"
|
||||
format_rsync_details "$DOWNLOAD_LOG_DB" "Download" "down"
|
||||
format_rsync_details "$DOWNLOAD_LOG_PHOTOS" "Download" "down"
|
||||
format_unison_details "$UNISON_LOG_DB" "Upload DB" "up"
|
||||
format_unison_details "$UNISON_LOG_DB" "Download DB" "down"
|
||||
format_unison_details "$UNISON_LOG_PHOTOS" "Upload Fotos" "up"
|
||||
format_unison_details "$UNISON_LOG_PHOTOS" "Download Fotos" "down"
|
||||
fi
|
||||
else
|
||||
log "Keine Änderungen – alles aktuell."
|
||||
fi
|
||||
else
|
||||
log_step "Sync abgeschlossen"
|
||||
log " Hochgeladen: $TOTAL_SENT ($SENT_DB DB + $SENT_PHOTOS Fotos)"
|
||||
log " Heruntergeladen: $TOTAL_RECEIVED ($RECEIVED_DB DB + $RECEIVED_PHOTOS Fotos)"
|
||||
log " Geändert: $TOTAL_CHANGED ($CHANGED_DB DB + $CHANGED_PHOTOS Fotos)"
|
||||
|
||||
if [ "$TOTAL_SENT" -gt 0 ] || [ "$TOTAL_RECEIVED" -gt 0 ]; then
|
||||
if [ "$TOTAL_CHANGED" -gt 0 ]; then
|
||||
notify-send "Darktable Sync" \
|
||||
"↑ $TOTAL_SENT hochgeladen | ↓ $TOTAL_RECEIVED heruntergeladen" -t 10000
|
||||
"↕ $TOTAL_CHANGED Datei(en) synchronisiert" -t 10000
|
||||
else
|
||||
log "Keine Änderungen – alles aktuell."
|
||||
fi
|
||||
|
||||
+36
-25
@@ -15,7 +15,7 @@ setup() {
|
||||
export DISPLAY=:99
|
||||
}
|
||||
|
||||
# --- Bestehende Tests (echter Sync via --execute) ---
|
||||
# --- Grundlegende Sync-Verhaltenstests ---
|
||||
|
||||
@test "sync_pending wird gesetzt wenn Server nicht erreichbar" {
|
||||
run_with_stubs env SSH_STUB_FAIL=1 bash "$SYNC_SCRIPT" --execute
|
||||
@@ -30,13 +30,13 @@ setup() {
|
||||
[ ! -f "$CONFIG_DIR/sync_pending" ]
|
||||
}
|
||||
|
||||
@test "sync_pending wird gesetzt wenn rsync fehlschlaegt" {
|
||||
run_with_stubs env SSH_STUB_FAIL=0 RSYNC_STUB_FAIL=1 bash "$SYNC_SCRIPT" --execute
|
||||
@test "sync_pending wird gesetzt wenn unison fehlschlaegt" {
|
||||
run_with_stubs env SSH_STUB_FAIL=0 UNISON_STUB_FAIL=1 bash "$SYNC_SCRIPT" --execute
|
||||
[ "$status" -eq 1 ]
|
||||
[ -f "$CONFIG_DIR/sync_pending" ]
|
||||
}
|
||||
|
||||
@test "DB-Backup wird vor Download erstellt" {
|
||||
@test "DB-Backup wird vor Sync erstellt" {
|
||||
run_with_stubs env SSH_STUB_FAIL=0 bash "$SYNC_SCRIPT" --execute
|
||||
[ "$status" -eq 0 ]
|
||||
[ -f "$HOME/.config/darktable/library.db.bak" ]
|
||||
@@ -62,7 +62,7 @@ setup() {
|
||||
[ ! -d "$CONFIG_DIR/sync.lock" ]
|
||||
}
|
||||
|
||||
# --- Neue Tests: Dry-Run-Verhalten ---
|
||||
# --- Dry-Run-Verhalten ---
|
||||
|
||||
@test "Trockenlauf ist Standard ohne --execute" {
|
||||
run_with_stubs env SSH_STUB_FAIL=0 DRY_RUN_SKIP_CONFIRM=1 bash "$SYNC_SCRIPT"
|
||||
@@ -106,29 +106,12 @@ setup() {
|
||||
|
||||
@test "Trockenlauf zaehlt neue Dateien korrekt" {
|
||||
run_with_stubs env SSH_STUB_FAIL=0 DRY_RUN_SKIP_CONFIRM=1 \
|
||||
RSYNC_STUB_DRY_LINES=">f+++++++++ foto.jpg" bash "$SYNC_SCRIPT"
|
||||
UNISON_STUB_DRY_LINES="new file ----> foto.jpg" bash "$SYNC_SCRIPT"
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"neu"* ]]
|
||||
}
|
||||
|
||||
# --- Tests: --delete und Backup-Verhalten ---
|
||||
|
||||
@test "Upload-rsync enthaelt --delete Flag" {
|
||||
ARGS_FILE=$(mktemp)
|
||||
run_with_stubs env SSH_STUB_FAIL=0 RSYNC_STUB_ARGS_FILE="$ARGS_FILE" bash "$SYNC_SCRIPT" --execute
|
||||
[ "$status" -eq 0 ]
|
||||
grep -q -- "--delete" "$ARGS_FILE"
|
||||
rm -f "$ARGS_FILE"
|
||||
}
|
||||
|
||||
@test "Download-rsync enthaelt --backup und --backup-dir" {
|
||||
ARGS_FILE=$(mktemp)
|
||||
run_with_stubs env SSH_STUB_FAIL=0 RSYNC_STUB_ARGS_FILE="$ARGS_FILE" bash "$SYNC_SCRIPT" --execute
|
||||
[ "$status" -eq 0 ]
|
||||
grep -q -- "--backup" "$ARGS_FILE"
|
||||
grep -q -- "--backup-dir=" "$ARGS_FILE"
|
||||
rm -f "$ARGS_FILE"
|
||||
}
|
||||
# --- Backup-Verhalten ---
|
||||
|
||||
@test "Backup-Verzeichnisse werden bei echtem Sync angelegt" {
|
||||
run_with_stubs env SSH_STUB_FAIL=0 bash "$SYNC_SCRIPT" --execute
|
||||
@@ -154,7 +137,35 @@ setup() {
|
||||
|
||||
@test "Trockenlauf-Ergebnis zeigt Backup-Hinweis bei Loeschungen" {
|
||||
run_with_stubs env SSH_STUB_FAIL=0 DRY_RUN_SKIP_CONFIRM=1 \
|
||||
RSYNC_STUB_DRY_LINES="*deleting foto.jpg" bash "$SYNC_SCRIPT"
|
||||
UNISON_STUB_DRY_LINES="<---- deleted foto.jpg" bash "$SYNC_SCRIPT"
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"Backup"* ]]
|
||||
}
|
||||
|
||||
# --- Unison-spezifische Tests ---
|
||||
|
||||
@test "Unison-Versionsmismatch gibt Exit 1 und setzt sync_pending" {
|
||||
run_with_stubs env SSH_STUB_FAIL=0 \
|
||||
UNISON_SERVER_VERSION="unison version 2.51.0" DRY_RUN_SKIP_CONFIRM=1 \
|
||||
bash "$SYNC_SCRIPT"
|
||||
[ "$status" -eq 1 ]
|
||||
[ -f "$CONFIG_DIR/sync_pending" ]
|
||||
}
|
||||
|
||||
@test "DB-Sync verwendet -force local bei Upload-Entscheidung" {
|
||||
echo "999" > "$CONFIG_DIR/sync_token"
|
||||
ARGS_FILE=$(mktemp)
|
||||
run_with_stubs env SSH_STUB_FAIL=0 UNISON_STUB_ARGS_FILE="$ARGS_FILE" \
|
||||
DARKTABLE_SYNC_CONFLICT_RESPONSE="upload" bash "$SYNC_SCRIPT" --execute
|
||||
grep -q -- "-force" "$ARGS_FILE" || grep -q -- "force" "$ARGS_FILE"
|
||||
rm -f "$ARGS_FILE"
|
||||
}
|
||||
|
||||
@test "Foto-Sync verwendet -prefer newer ohne Konflikt" {
|
||||
ARGS_FILE=$(mktemp)
|
||||
run_with_stubs env SSH_STUB_FAIL=0 UNISON_STUB_ARGS_FILE="$ARGS_FILE" \
|
||||
bash "$SYNC_SCRIPT" --execute
|
||||
[ "$status" -eq 0 ]
|
||||
grep -q -- "-prefer" "$ARGS_FILE"
|
||||
rm -f "$ARGS_FILE"
|
||||
}
|
||||
|
||||
+12
-12
@@ -185,24 +185,24 @@ EOF
|
||||
[ "$output" = "Sonstiges" ]
|
||||
}
|
||||
|
||||
@test "security: format_rsync_details mit nicht existierender Datei gibt nichts aus" {
|
||||
run bash -c "source '$COMMON_SCRIPT'; format_rsync_details '/tmp/nonexistent_$RANDOM' 'Upload' 'up'"
|
||||
@test "security: format_unison_details mit nicht existierender Datei gibt nichts aus" {
|
||||
run bash -c "source '$COMMON_SCRIPT'; format_unison_details '/tmp/nonexistent_$RANDOM' 'Upload' 'up'"
|
||||
[ "$status" -eq 0 ]
|
||||
[ -z "$output" ]
|
||||
}
|
||||
|
||||
@test "security: format_rsync_details mit manipulierten Log-Zeilen fuehrt keinen Code aus" {
|
||||
@test "security: format_unison_details mit manipulierten Log-Zeilen fuehrt keinen Code aus" {
|
||||
local evil_log
|
||||
evil_log=$(mktemp)
|
||||
# Zeile die aussieht wie rsync-Output aber Shell-Metazeichen enthaelt
|
||||
echo '>f+++++++++ $(touch /tmp/evil_rsync).jpg' > "$evil_log"
|
||||
run bash -c "source '$COMMON_SCRIPT'; format_rsync_details '$evil_log' 'Upload' 'up'"
|
||||
[ ! -f /tmp/evil_rsync ]
|
||||
# Zeile die aussieht wie Unison-Output aber Shell-Metazeichen enthaelt
|
||||
echo 'new file ----> $(touch /tmp/evil_unison).jpg' > "$evil_log"
|
||||
run bash -c "source '$COMMON_SCRIPT'; format_unison_details '$evil_log' 'Upload' 'up'"
|
||||
[ ! -f /tmp/evil_unison ]
|
||||
rm -f "$evil_log"
|
||||
}
|
||||
|
||||
@test "security: RSYNC_DRY_FLAG ist leer bei --execute" {
|
||||
# Verifiziere dass bei --execute kein --dry-run an rsync uebergeben wird
|
||||
@test "security: UNISON_DRY_FLAG ist leer bei --execute" {
|
||||
# Verifiziere dass bei --execute kein -dryrun an unison uebergeben wird
|
||||
run_with_stubs env SSH_STUB_FAIL=0 bash "$SYNC_SCRIPT" --execute
|
||||
[ "$status" -eq 0 ]
|
||||
# Backup muss existieren (nur bei echtem Sync)
|
||||
@@ -235,10 +235,10 @@ EOF
|
||||
[[ "$output" == *"TROCKENLAUF"* ]]
|
||||
}
|
||||
|
||||
@test "security: echo -e im rsync-Stub fuehrt keinen Code aus" {
|
||||
# RSYNC_STUB_DRY_LINES mit Shell-Metazeichen
|
||||
@test "security: Unison-Stub mit Shell-Metazeichen fuehrt keinen Code aus" {
|
||||
# UNISON_STUB_DRY_LINES mit Shell-Metazeichen
|
||||
run_with_stubs env SSH_STUB_FAIL=0 DRY_RUN_SKIP_CONFIRM=1 \
|
||||
RSYNC_STUB_DRY_LINES='>f+++++++++ $(touch /tmp/evil_stub).jpg' bash "$SYNC_SCRIPT"
|
||||
UNISON_STUB_DRY_LINES='new file ----> $(touch /tmp/evil_stub).jpg' bash "$SYNC_SCRIPT"
|
||||
[ "$status" -eq 0 ]
|
||||
[ ! -f /tmp/evil_stub ]
|
||||
}
|
||||
|
||||
+10
-2
@@ -1,7 +1,15 @@
|
||||
#!/bin/bash
|
||||
# SSH_STUB_FAIL=1 → schlaegt fehl
|
||||
# SSH_STUB_FAIL=1 → schlägt fehl
|
||||
# SSH_STUB_OUTPUT → Ausgabe für allgemeine Befehle
|
||||
# UNISON_SERVER_VERSION → Ausgabe für 'unison -version' (Standard: passt zur lokalen Version)
|
||||
|
||||
if [ "${SSH_STUB_FAIL:-0}" = "1" ]; then
|
||||
exit 1
|
||||
fi
|
||||
echo "${SSH_STUB_OUTPUT:-}"
|
||||
|
||||
if echo "$*" | grep -q 'unison -version'; then
|
||||
echo "${UNISON_SERVER_VERSION:-unison version 2.53.3}"
|
||||
else
|
||||
echo "${SSH_STUB_OUTPUT:-}"
|
||||
fi
|
||||
exit 0
|
||||
|
||||
Reference in New Issue
Block a user