ZFS incremental snapshot send/recv without first snapshot












2















I have the following ZFS dataset:



pool/dataset

pool/dataset@snap1

pool/dataset@snap2

pool/dataset@snap3


Which has been replicated to a backup pool, using ZFS send/recv



backupPool/dataset

backupPool/dataset@snap1

backupPool/dataset@snap2


Afterwards, I deleted dataset@snap1 and dataset@snap2 from pool, and I am in a situation where I cannot send dataset@snap3 incrementally to backupPool.



Is there a way of solving this situation? For instance, generate a ZFS incremental snapshot between pool/dataset@snap3 and backupPool/dataset@snap2 and send it to backupPool? Or transfer back backupPool/dataset@snap2 to pool?



I could transfer pool/dataset@snap3 to a new dataset in backupPool, but I really need to keep the "history" of snapshots.






zfs snapshot







share|improve this question























  • Do you really need to have a ZFS dataset as a destination for your backups? It makes the whole concept difficult to manage. I have a similar setup for my backups. I just use serialized compressed snapshots with multiple incremental levels.

    – Martin Sugioarto
    Mar 6 at 7:56











  • Yes, I do need a ZFS dataset with the backup.

    – fclad
    Mar 6 at 15:24











  • @MartinSugioarto serialized compressed snapshots with multiple incremental levels That does leave you vulnerable to losing your backups entirely if you later have a problem with zfs receive. One advantage of immediately running zfs receive ... is that any issue is known immediately - not with you have to restore.

    – Andrew Henle
    Mar 7 at 10:34













  • Serialized snapshots can be only a problem, if you delete them in wrong order. Level 0 snapshots are most important, of course. The problem with live datasets is that if something accidentally writes to such a dataset, it's a totally different dataset from the logical point of view and you need to roll back. And if you delete a snapshot there, it's also a write access. On the sending side, if you delete a snapshot, all parent snapshots correct their diffs up to the top live filesystem and you lose any connection with you receiving dataset. This method appears very unstable to me.

    – Martin Sugioarto
    Mar 7 at 10:45






  • 1





    I subtlely misunderstood the question! @AndrewHenle's approach would work. When sending the new snapshot back to pool, you won't be able to do an incremental send, you just have to do a full send from scratch and create a new filesystem. By the way, if you deleted pool/dataset@snap2 to save space, you could try using bookmarks on the sending system instead of full snapshots -- they let ZFS know what changed since the last time you did a send, without keeping all of the data from that last snapshot around forever.

    – Dan
    Mar 8 at 3:08
















2















I have the following ZFS dataset:



pool/dataset

pool/dataset@snap1

pool/dataset@snap2

pool/dataset@snap3


Which has been replicated to a backup pool, using ZFS send/recv



backupPool/dataset

backupPool/dataset@snap1

backupPool/dataset@snap2


Afterwards, I deleted dataset@snap1 and dataset@snap2 from pool, and I am in a situation where I cannot send dataset@snap3 incrementally to backupPool.



Is there a way of solving this situation? For instance, generate a ZFS incremental snapshot between pool/dataset@snap3 and backupPool/dataset@snap2 and send it to backupPool? Or transfer back backupPool/dataset@snap2 to pool?



I could transfer pool/dataset@snap3 to a new dataset in backupPool, but I really need to keep the "history" of snapshots.






zfs snapshot







share|improve this question























  • Do you really need to have a ZFS dataset as a destination for your backups? It makes the whole concept difficult to manage. I have a similar setup for my backups. I just use serialized compressed snapshots with multiple incremental levels.

    – Martin Sugioarto
    Mar 6 at 7:56











  • Yes, I do need a ZFS dataset with the backup.

    – fclad
    Mar 6 at 15:24











  • @MartinSugioarto serialized compressed snapshots with multiple incremental levels That does leave you vulnerable to losing your backups entirely if you later have a problem with zfs receive. One advantage of immediately running zfs receive ... is that any issue is known immediately - not with you have to restore.

    – Andrew Henle
    Mar 7 at 10:34













  • Serialized snapshots can be only a problem, if you delete them in wrong order. Level 0 snapshots are most important, of course. The problem with live datasets is that if something accidentally writes to such a dataset, it's a totally different dataset from the logical point of view and you need to roll back. And if you delete a snapshot there, it's also a write access. On the sending side, if you delete a snapshot, all parent snapshots correct their diffs up to the top live filesystem and you lose any connection with you receiving dataset. This method appears very unstable to me.

    – Martin Sugioarto
    Mar 7 at 10:45






  • 1





    I subtlely misunderstood the question! @AndrewHenle's approach would work. When sending the new snapshot back to pool, you won't be able to do an incremental send, you just have to do a full send from scratch and create a new filesystem. By the way, if you deleted pool/dataset@snap2 to save space, you could try using bookmarks on the sending system instead of full snapshots -- they let ZFS know what changed since the last time you did a send, without keeping all of the data from that last snapshot around forever.

    – Dan
    Mar 8 at 3:08














2












2








2








I have the following ZFS dataset:



pool/dataset

pool/dataset@snap1

pool/dataset@snap2

pool/dataset@snap3


Which has been replicated to a backup pool, using ZFS send/recv



backupPool/dataset

backupPool/dataset@snap1

backupPool/dataset@snap2


Afterwards, I deleted dataset@snap1 and dataset@snap2 from pool, and I am in a situation where I cannot send dataset@snap3 incrementally to backupPool.



Is there a way of solving this situation? For instance, generate a ZFS incremental snapshot between pool/dataset@snap3 and backupPool/dataset@snap2 and send it to backupPool? Or transfer back backupPool/dataset@snap2 to pool?



I could transfer pool/dataset@snap3 to a new dataset in backupPool, but I really need to keep the "history" of snapshots.






zfs snapshot







share|improve this question














I have the following ZFS dataset:



pool/dataset

pool/dataset@snap1

pool/dataset@snap2

pool/dataset@snap3


Which has been replicated to a backup pool, using ZFS send/recv



backupPool/dataset

backupPool/dataset@snap1

backupPool/dataset@snap2


Afterwards, I deleted dataset@snap1 and dataset@snap2 from pool, and I am in a situation where I cannot send dataset@snap3 incrementally to backupPool.



Is there a way of solving this situation? For instance, generate a ZFS incremental snapshot between pool/dataset@snap3 and backupPool/dataset@snap2 and send it to backupPool? Or transfer back backupPool/dataset@snap2 to pool?



I could transfer pool/dataset@snap3 to a new dataset in backupPool, but I really need to keep the "history" of snapshots.






zfs snapshot




zfs snapshot






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Mar 6 at 0:01









fcladfclad

1687




1687













  • Do you really need to have a ZFS dataset as a destination for your backups? It makes the whole concept difficult to manage. I have a similar setup for my backups. I just use serialized compressed snapshots with multiple incremental levels.

    – Martin Sugioarto
    Mar 6 at 7:56











  • Yes, I do need a ZFS dataset with the backup.

    – fclad
    Mar 6 at 15:24











  • @MartinSugioarto serialized compressed snapshots with multiple incremental levels That does leave you vulnerable to losing your backups entirely if you later have a problem with zfs receive. One advantage of immediately running zfs receive ... is that any issue is known immediately - not with you have to restore.

    – Andrew Henle
    Mar 7 at 10:34













  • Serialized snapshots can be only a problem, if you delete them in wrong order. Level 0 snapshots are most important, of course. The problem with live datasets is that if something accidentally writes to such a dataset, it's a totally different dataset from the logical point of view and you need to roll back. And if you delete a snapshot there, it's also a write access. On the sending side, if you delete a snapshot, all parent snapshots correct their diffs up to the top live filesystem and you lose any connection with you receiving dataset. This method appears very unstable to me.

    – Martin Sugioarto
    Mar 7 at 10:45






  • 1





    I subtlely misunderstood the question! @AndrewHenle's approach would work. When sending the new snapshot back to pool, you won't be able to do an incremental send, you just have to do a full send from scratch and create a new filesystem. By the way, if you deleted pool/dataset@snap2 to save space, you could try using bookmarks on the sending system instead of full snapshots -- they let ZFS know what changed since the last time you did a send, without keeping all of the data from that last snapshot around forever.

    – Dan
    Mar 8 at 3:08



















  • Do you really need to have a ZFS dataset as a destination for your backups? It makes the whole concept difficult to manage. I have a similar setup for my backups. I just use serialized compressed snapshots with multiple incremental levels.

    – Martin Sugioarto
    Mar 6 at 7:56











  • Yes, I do need a ZFS dataset with the backup.

    – fclad
    Mar 6 at 15:24











  • @MartinSugioarto serialized compressed snapshots with multiple incremental levels That does leave you vulnerable to losing your backups entirely if you later have a problem with zfs receive. One advantage of immediately running zfs receive ... is that any issue is known immediately - not with you have to restore.

    – Andrew Henle
    Mar 7 at 10:34













  • Serialized snapshots can be only a problem, if you delete them in wrong order. Level 0 snapshots are most important, of course. The problem with live datasets is that if something accidentally writes to such a dataset, it's a totally different dataset from the logical point of view and you need to roll back. And if you delete a snapshot there, it's also a write access. On the sending side, if you delete a snapshot, all parent snapshots correct their diffs up to the top live filesystem and you lose any connection with you receiving dataset. This method appears very unstable to me.

    – Martin Sugioarto
    Mar 7 at 10:45






  • 1





    I subtlely misunderstood the question! @AndrewHenle's approach would work. When sending the new snapshot back to pool, you won't be able to do an incremental send, you just have to do a full send from scratch and create a new filesystem. By the way, if you deleted pool/dataset@snap2 to save space, you could try using bookmarks on the sending system instead of full snapshots -- they let ZFS know what changed since the last time you did a send, without keeping all of the data from that last snapshot around forever.

    – Dan
    Mar 8 at 3:08

















Do you really need to have a ZFS dataset as a destination for your backups? It makes the whole concept difficult to manage. I have a similar setup for my backups. I just use serialized compressed snapshots with multiple incremental levels.

– Martin Sugioarto
Mar 6 at 7:56





Do you really need to have a ZFS dataset as a destination for your backups? It makes the whole concept difficult to manage. I have a similar setup for my backups. I just use serialized compressed snapshots with multiple incremental levels.

– Martin Sugioarto
Mar 6 at 7:56













Yes, I do need a ZFS dataset with the backup.

– fclad
Mar 6 at 15:24





Yes, I do need a ZFS dataset with the backup.

– fclad
Mar 6 at 15:24













@MartinSugioarto serialized compressed snapshots with multiple incremental levels That does leave you vulnerable to losing your backups entirely if you later have a problem with zfs receive. One advantage of immediately running zfs receive ... is that any issue is known immediately - not with you have to restore.

– Andrew Henle
Mar 7 at 10:34







@MartinSugioarto serialized compressed snapshots with multiple incremental levels That does leave you vulnerable to losing your backups entirely if you later have a problem with zfs receive. One advantage of immediately running zfs receive ... is that any issue is known immediately - not with you have to restore.

– Andrew Henle
Mar 7 at 10:34















Serialized snapshots can be only a problem, if you delete them in wrong order. Level 0 snapshots are most important, of course. The problem with live datasets is that if something accidentally writes to such a dataset, it's a totally different dataset from the logical point of view and you need to roll back. And if you delete a snapshot there, it's also a write access. On the sending side, if you delete a snapshot, all parent snapshots correct their diffs up to the top live filesystem and you lose any connection with you receiving dataset. This method appears very unstable to me.

– Martin Sugioarto
Mar 7 at 10:45





Serialized snapshots can be only a problem, if you delete them in wrong order. Level 0 snapshots are most important, of course. The problem with live datasets is that if something accidentally writes to such a dataset, it's a totally different dataset from the logical point of view and you need to roll back. And if you delete a snapshot there, it's also a write access. On the sending side, if you delete a snapshot, all parent snapshots correct their diffs up to the top live filesystem and you lose any connection with you receiving dataset. This method appears very unstable to me.

– Martin Sugioarto
Mar 7 at 10:45




1




1





I subtlely misunderstood the question! @AndrewHenle's approach would work. When sending the new snapshot back to pool, you won't be able to do an incremental send, you just have to do a full send from scratch and create a new filesystem. By the way, if you deleted pool/dataset@snap2 to save space, you could try using bookmarks on the sending system instead of full snapshots -- they let ZFS know what changed since the last time you did a send, without keeping all of the data from that last snapshot around forever.

– Dan
Mar 8 at 3:08





I subtlely misunderstood the question! @AndrewHenle's approach would work. When sending the new snapshot back to pool, you won't be able to do an incremental send, you just have to do a full send from scratch and create a new filesystem. By the way, if you deleted pool/dataset@snap2 to save space, you could try using bookmarks on the sending system instead of full snapshots -- they let ZFS know what changed since the last time you did a send, without keeping all of the data from that last snapshot around forever.

– Dan
Mar 8 at 3:08










1 Answer
1






active

oldest

votes


















0














Thank you for all your suggestions!



I finally rsynced pool/dataset@snap3 with backupPool/dataset@snap2, deleted the dataset backupPool/dataset, and recreated it from backupPool/dataset. I was not able to find a better solution to this problem.



The suggestion from Dan was really helpful. Also, to avoid deleting snapshots in the future it is a good practice to hold them.






share|improve this answer
























    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "106"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f504595%2fzfs-incremental-snapshot-send-recv-without-first-snapshot%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    Thank you for all your suggestions!



    I finally rsynced pool/dataset@snap3 with backupPool/dataset@snap2, deleted the dataset backupPool/dataset, and recreated it from backupPool/dataset. I was not able to find a better solution to this problem.



    The suggestion from Dan was really helpful. Also, to avoid deleting snapshots in the future it is a good practice to hold them.






    share|improve this answer




























      0














      Thank you for all your suggestions!



      I finally rsynced pool/dataset@snap3 with backupPool/dataset@snap2, deleted the dataset backupPool/dataset, and recreated it from backupPool/dataset. I was not able to find a better solution to this problem.



      The suggestion from Dan was really helpful. Also, to avoid deleting snapshots in the future it is a good practice to hold them.






      share|improve this answer


























        0












        0








        0







        Thank you for all your suggestions!



        I finally rsynced pool/dataset@snap3 with backupPool/dataset@snap2, deleted the dataset backupPool/dataset, and recreated it from backupPool/dataset. I was not able to find a better solution to this problem.



        The suggestion from Dan was really helpful. Also, to avoid deleting snapshots in the future it is a good practice to hold them.






        share|improve this answer













        Thank you for all your suggestions!



        I finally rsynced pool/dataset@snap3 with backupPool/dataset@snap2, deleted the dataset backupPool/dataset, and recreated it from backupPool/dataset. I was not able to find a better solution to this problem.



        The suggestion from Dan was really helpful. Also, to avoid deleting snapshots in the future it is a good practice to hold them.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Mar 8 at 17:31









        fcladfclad

        1687




        1687






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Unix & Linux Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f504595%2fzfs-incremental-snapshot-send-recv-without-first-snapshot%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            How to make a Squid Proxy server?

            Image
            1 I want to set up squid proxy server for monitoring user activity in a network and Internet Web URL filtering for user. I also want to configure the user login, user log, user access group, and block unwanted web sites in my network. Please help me and guide me the full process to do this. server proxy squid share | improve this question edited Jul 31 '12 at 14:45 Jorge Castro 37.1k 107 422 617 asked Jul 31 '12 at 5:16 Goivind Tiwari Goivind Tiwari ...
            Read more

            Is this a new Fibonacci Identity?

            Image
            6 1 $begingroup$ I have found the following Fibonacci Identity (and proved it). If $F_n$ denotes the nth Fibonacci Number, we have the following identity begin{equation} F_{n-r+h}F_{n+k+g+1} - F_{n-r+g}F_{n+k+h+1} = (-1)^{n+r+h+1} F_{g-h}F_{k+r+1} end{equation} where $F_1 = F_2 = 1$ , $r leq n$ , $h leq g$ , and $n, g, k in mathbb{N}$ . It is not too hard to show that this identity subsumes Cassini's Identity, Catalan's Identity, Vajda's Idenity, and d'Ocagne's identity to name a few. I have done a pretty thorough literature review, and I have not found anything like this, but I am still wondering if anyone has seen this identity before? I found this by accident after noticing some patterns in some analysis work I was doing, so if this is already known I would be curious to see w...
            Read more

            19世紀

            Image
            この記事は検証可能な参考文献や出典が全く示されていないか、不十分です。 出典を追加して記事の信頼性向上にご協力ください。 （ 2012年3月 ） 千年紀: 2千年紀 世紀: 18世紀 - 19世紀 - 20世紀 十年紀: 1800年代 1810年代 1820年代 1830年代 1840年代 1850年代 1860年代 1870年代 1880年代 1890年代 19世紀に君臨した大英帝国。 19世紀 （じゅうきゅうせいき）は、西暦1801年から西暦1900年までの100年間を指す世紀。 目次 1 19世紀の歴史 1.1 国民国家の成立 1.2 帝国主義の興隆 1.3 列強の植民地争奪戦 2 できごと 2.1 1800年代 2.2 1810年代 2.3 1820年代 2.4 1830年代 2.5 1840年代 2.6 1850年代 2.7 1860年代 2.8 1870年代 2.9 1880年代 2.10 1890年代 2.11 1900年代 3 文化 3.1 文学 3.2 音楽 3.3 思想 3.4 科学 3.5 技術 4 人物 4.1 ヨーロッパ 4.1.1 政治家・王族 4.1.1.1 フランス 4.1.1.2 オーストリア＝ハンガリー 4.1.1.3 ロシア 4.1.1.4 イギリス 4.1.1.5 ドイツ（プロイセンほかドイツ領邦を含む） 4.1.1.6 北欧 4.1.1.7 イタリア 4.1.1.8 スペイン 4.1.1.9 ベルギー 4.1.1.10 ギリシア 4.1.2 軍人 4.1.3 実業家 4.1.4 科学と技術 4.1.5 思想と哲学・人文諸学 4.1.6 宗教 4.1.7 文学 4.1.8 美術 4.1.9 音楽 4.1.10 社会事業家 4.1.11 探検家・旅行家 4.1.12 料...
            Read more