is 'sed' thread safe












2















If I have a shell/python script that uses sed to modify a file in place based on user inputs, and then two users run the same script at the same time or approx. same time, is 'sed' thread safe ? Or perhaps it is not an issue because the file_descripor that was opened by the first thread will be used to lock the file anyway ? thx






linux sed python







share|improve this question




















  • 2





    Slightly more info wanted. How do you use sed from Python (and why, can't Python do things like that fairly effortlessly?).

    – Kusalananda
    7 hours ago
















2















If I have a shell/python script that uses sed to modify a file in place based on user inputs, and then two users run the same script at the same time or approx. same time, is 'sed' thread safe ? Or perhaps it is not an issue because the file_descripor that was opened by the first thread will be used to lock the file anyway ? thx






linux sed python







share|improve this question




















  • 2





    Slightly more info wanted. How do you use sed from Python (and why, can't Python do things like that fairly effortlessly?).

    – Kusalananda
    7 hours ago














2












2








2








If I have a shell/python script that uses sed to modify a file in place based on user inputs, and then two users run the same script at the same time or approx. same time, is 'sed' thread safe ? Or perhaps it is not an issue because the file_descripor that was opened by the first thread will be used to lock the file anyway ? thx






linux sed python







share|improve this question
















If I have a shell/python script that uses sed to modify a file in place based on user inputs, and then two users run the same script at the same time or approx. same time, is 'sed' thread safe ? Or perhaps it is not an issue because the file_descripor that was opened by the first thread will be used to lock the file anyway ? thx






linux sed python




linux sed python






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 7 hours ago









Jeff Schaller

42.9k1159137




42.9k1159137










asked 7 hours ago









terreysterreys

1613




1613








  • 2





    Slightly more info wanted. How do you use sed from Python (and why, can't Python do things like that fairly effortlessly?).

    – Kusalananda
    7 hours ago














  • 2





    Slightly more info wanted. How do you use sed from Python (and why, can't Python do things like that fairly effortlessly?).

    – Kusalananda
    7 hours ago








2




2





Slightly more info wanted. How do you use sed from Python (and why, can't Python do things like that fairly effortlessly?).

– Kusalananda
7 hours ago





Slightly more info wanted. How do you use sed from Python (and why, can't Python do things like that fairly effortlessly?).

– Kusalananda
7 hours ago










1 Answer
1






active

oldest

votes


















6














I'm not going to nitpick on the awful terminology, but yes, GNU sed with its -i ("in-place") flag could be safely used by more than one process at the same time without any extra locking, because sed is not actually modifying the file in-place, but it's redirecting the output to a temporary file, and if everything goes well, it will rename(2) (move) the temporary file to the original file, and the rename(2) is guaranteed to be atomic:



$ strace sed -i s/o/e/g foo.txt

open("foo.txt", O_RDONLY)               = 3

...

open("./sedDe80VL", O_RDWR|O_CREAT|O_EXCL, 0600) = 4

...

read(3, "foon", 4096)                  = 4

...

write(4, "feen", 4)                    = 4

read(3, "", 4096)                       = 0

...

close(3)                                = 0

close(4)                                = 0

rename("./sedDe80VL", "foo.txt")        = 0


At any point, foo.txt will refer either to the complete original file or to the complete processed file, never to something in between the two.






share|improve this answer





















  • 3





    But what can happen is that two copies of sed start reading the file, make different changes, storing them to their respective temporary files, which are then renamed into place one after other, without regard for the fact that there was another sed working on the file at the same time. The changes made by one of the sed processes would be lost.

    – ilkkachu
    6 hours ago








  • 2





    @ilkkachu that will still be completely consistent -- the file will be either modified by both processes in turn (in any order) or by just one of them. At no point will the file contain garbage resulting from both processes modifying it at the same time.

    – mosvy
    6 hours ago






  • 7





    it won't be total garbage, but having changes lost can still be an issue. The question refers to the file "being locked", and locking would usually refer to the second sed process waiting until the first completed. That might just be awful terminology on their part; it's hard to say what they really care about, but that particular issue is still possible.

    – ilkkachu
    6 hours ago











Your Answer








StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f504810%2fis-sed-thread-safe%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









6














I'm not going to nitpick on the awful terminology, but yes, GNU sed with its -i ("in-place") flag could be safely used by more than one process at the same time without any extra locking, because sed is not actually modifying the file in-place, but it's redirecting the output to a temporary file, and if everything goes well, it will rename(2) (move) the temporary file to the original file, and the rename(2) is guaranteed to be atomic:



$ strace sed -i s/o/e/g foo.txt

open("foo.txt", O_RDONLY)               = 3

...

open("./sedDe80VL", O_RDWR|O_CREAT|O_EXCL, 0600) = 4

...

read(3, "foon", 4096)                  = 4

...

write(4, "feen", 4)                    = 4

read(3, "", 4096)                       = 0

...

close(3)                                = 0

close(4)                                = 0

rename("./sedDe80VL", "foo.txt")        = 0


At any point, foo.txt will refer either to the complete original file or to the complete processed file, never to something in between the two.






share|improve this answer





















  • 3





    But what can happen is that two copies of sed start reading the file, make different changes, storing them to their respective temporary files, which are then renamed into place one after other, without regard for the fact that there was another sed working on the file at the same time. The changes made by one of the sed processes would be lost.

    – ilkkachu
    6 hours ago








  • 2





    @ilkkachu that will still be completely consistent -- the file will be either modified by both processes in turn (in any order) or by just one of them. At no point will the file contain garbage resulting from both processes modifying it at the same time.

    – mosvy
    6 hours ago






  • 7





    it won't be total garbage, but having changes lost can still be an issue. The question refers to the file "being locked", and locking would usually refer to the second sed process waiting until the first completed. That might just be awful terminology on their part; it's hard to say what they really care about, but that particular issue is still possible.

    – ilkkachu
    6 hours ago
















6














I'm not going to nitpick on the awful terminology, but yes, GNU sed with its -i ("in-place") flag could be safely used by more than one process at the same time without any extra locking, because sed is not actually modifying the file in-place, but it's redirecting the output to a temporary file, and if everything goes well, it will rename(2) (move) the temporary file to the original file, and the rename(2) is guaranteed to be atomic:



$ strace sed -i s/o/e/g foo.txt

open("foo.txt", O_RDONLY)               = 3

...

open("./sedDe80VL", O_RDWR|O_CREAT|O_EXCL, 0600) = 4

...

read(3, "foon", 4096)                  = 4

...

write(4, "feen", 4)                    = 4

read(3, "", 4096)                       = 0

...

close(3)                                = 0

close(4)                                = 0

rename("./sedDe80VL", "foo.txt")        = 0


At any point, foo.txt will refer either to the complete original file or to the complete processed file, never to something in between the two.






share|improve this answer





















  • 3





    But what can happen is that two copies of sed start reading the file, make different changes, storing them to their respective temporary files, which are then renamed into place one after other, without regard for the fact that there was another sed working on the file at the same time. The changes made by one of the sed processes would be lost.

    – ilkkachu
    6 hours ago








  • 2





    @ilkkachu that will still be completely consistent -- the file will be either modified by both processes in turn (in any order) or by just one of them. At no point will the file contain garbage resulting from both processes modifying it at the same time.

    – mosvy
    6 hours ago






  • 7





    it won't be total garbage, but having changes lost can still be an issue. The question refers to the file "being locked", and locking would usually refer to the second sed process waiting until the first completed. That might just be awful terminology on their part; it's hard to say what they really care about, but that particular issue is still possible.

    – ilkkachu
    6 hours ago














6












6








6







I'm not going to nitpick on the awful terminology, but yes, GNU sed with its -i ("in-place") flag could be safely used by more than one process at the same time without any extra locking, because sed is not actually modifying the file in-place, but it's redirecting the output to a temporary file, and if everything goes well, it will rename(2) (move) the temporary file to the original file, and the rename(2) is guaranteed to be atomic:



$ strace sed -i s/o/e/g foo.txt

open("foo.txt", O_RDONLY)               = 3

...

open("./sedDe80VL", O_RDWR|O_CREAT|O_EXCL, 0600) = 4

...

read(3, "foon", 4096)                  = 4

...

write(4, "feen", 4)                    = 4

read(3, "", 4096)                       = 0

...

close(3)                                = 0

close(4)                                = 0

rename("./sedDe80VL", "foo.txt")        = 0


At any point, foo.txt will refer either to the complete original file or to the complete processed file, never to something in between the two.






share|improve this answer















I'm not going to nitpick on the awful terminology, but yes, GNU sed with its -i ("in-place") flag could be safely used by more than one process at the same time without any extra locking, because sed is not actually modifying the file in-place, but it's redirecting the output to a temporary file, and if everything goes well, it will rename(2) (move) the temporary file to the original file, and the rename(2) is guaranteed to be atomic:



$ strace sed -i s/o/e/g foo.txt

open("foo.txt", O_RDONLY)               = 3

...

open("./sedDe80VL", O_RDWR|O_CREAT|O_EXCL, 0600) = 4

...

read(3, "foon", 4096)                  = 4

...

write(4, "feen", 4)                    = 4

read(3, "", 4096)                       = 0

...

close(3)                                = 0

close(4)                                = 0

rename("./sedDe80VL", "foo.txt")        = 0


At any point, foo.txt will refer either to the complete original file or to the complete processed file, never to something in between the two.







share|improve this answer














share|improve this answer



share|improve this answer








edited 7 hours ago

























answered 7 hours ago









mosvymosvy

7,8021530




7,8021530








  • 3





    But what can happen is that two copies of sed start reading the file, make different changes, storing them to their respective temporary files, which are then renamed into place one after other, without regard for the fact that there was another sed working on the file at the same time. The changes made by one of the sed processes would be lost.

    – ilkkachu
    6 hours ago








  • 2





    @ilkkachu that will still be completely consistent -- the file will be either modified by both processes in turn (in any order) or by just one of them. At no point will the file contain garbage resulting from both processes modifying it at the same time.

    – mosvy
    6 hours ago






  • 7





    it won't be total garbage, but having changes lost can still be an issue. The question refers to the file "being locked", and locking would usually refer to the second sed process waiting until the first completed. That might just be awful terminology on their part; it's hard to say what they really care about, but that particular issue is still possible.

    – ilkkachu
    6 hours ago














  • 3





    But what can happen is that two copies of sed start reading the file, make different changes, storing them to their respective temporary files, which are then renamed into place one after other, without regard for the fact that there was another sed working on the file at the same time. The changes made by one of the sed processes would be lost.

    – ilkkachu
    6 hours ago








  • 2





    @ilkkachu that will still be completely consistent -- the file will be either modified by both processes in turn (in any order) or by just one of them. At no point will the file contain garbage resulting from both processes modifying it at the same time.

    – mosvy
    6 hours ago






  • 7





    it won't be total garbage, but having changes lost can still be an issue. The question refers to the file "being locked", and locking would usually refer to the second sed process waiting until the first completed. That might just be awful terminology on their part; it's hard to say what they really care about, but that particular issue is still possible.

    – ilkkachu
    6 hours ago








3




3





But what can happen is that two copies of sed start reading the file, make different changes, storing them to their respective temporary files, which are then renamed into place one after other, without regard for the fact that there was another sed working on the file at the same time. The changes made by one of the sed processes would be lost.

– ilkkachu
6 hours ago







But what can happen is that two copies of sed start reading the file, make different changes, storing them to their respective temporary files, which are then renamed into place one after other, without regard for the fact that there was another sed working on the file at the same time. The changes made by one of the sed processes would be lost.

– ilkkachu
6 hours ago






2




2





@ilkkachu that will still be completely consistent -- the file will be either modified by both processes in turn (in any order) or by just one of them. At no point will the file contain garbage resulting from both processes modifying it at the same time.

– mosvy
6 hours ago





@ilkkachu that will still be completely consistent -- the file will be either modified by both processes in turn (in any order) or by just one of them. At no point will the file contain garbage resulting from both processes modifying it at the same time.

– mosvy
6 hours ago




7




7





it won't be total garbage, but having changes lost can still be an issue. The question refers to the file "being locked", and locking would usually refer to the second sed process waiting until the first completed. That might just be awful terminology on their part; it's hard to say what they really care about, but that particular issue is still possible.

– ilkkachu
6 hours ago





it won't be total garbage, but having changes lost can still be an issue. The question refers to the file "being locked", and locking would usually refer to the second sed process waiting until the first completed. That might just be awful terminology on their part; it's hard to say what they really care about, but that particular issue is still possible.

– ilkkachu
6 hours ago


















draft saved

draft discarded




















































Thanks for contributing an answer to Unix & Linux Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid …



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f504810%2fis-sed-thread-safe%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







-

Popular posts from this blog

How to make a Squid Proxy server?

Image
1 I want to set up squid proxy server for monitoring user activity in a network and Internet Web URL filtering for user. I also want to configure the user login, user log, user access group, and block unwanted web sites in my network. Please help me and guide me the full process to do this. server proxy squid share | improve this question edited Jul 31 '12 at 14:45 Jorge Castro 37.1k 107 422 617 asked Jul 31 '12 at 5:16 Goivind Tiwari Goivind Tiwari ...
Read more

Is this a new Fibonacci Identity?

Image
6 1 $begingroup$ I have found the following Fibonacci Identity (and proved it). If $F_n$ denotes the nth Fibonacci Number, we have the following identity begin{equation} F_{n-r+h}F_{n+k+g+1} - F_{n-r+g}F_{n+k+h+1} = (-1)^{n+r+h+1} F_{g-h}F_{k+r+1} end{equation} where $F_1 = F_2 = 1$ , $r leq n$ , $h leq g$ , and $n, g, k in mathbb{N}$ . It is not too hard to show that this identity subsumes Cassini's Identity, Catalan's Identity, Vajda's Idenity, and d'Ocagne's identity to name a few. I have done a pretty thorough literature review, and I have not found anything like this, but I am still wondering if anyone has seen this identity before? I found this by accident after noticing some patterns in some analysis work I was doing, so if this is already known I would be curious to see w...
Read more

19世紀

Image
この記事は検証可能な参考文献や出典が全く示されていないか、不十分です。 出典を追加して記事の信頼性向上にご協力ください。 （ 2012年3月 ） 千年紀: 2千年紀 世紀: 18世紀 - 19世紀 - 20世紀 十年紀: 1800年代 1810年代 1820年代 1830年代 1840年代 1850年代 1860年代 1870年代 1880年代 1890年代 19世紀に君臨した大英帝国。 19世紀 （じゅうきゅうせいき）は、西暦1801年から西暦1900年までの100年間を指す世紀。 目次 1 19世紀の歴史 1.1 国民国家の成立 1.2 帝国主義の興隆 1.3 列強の植民地争奪戦 2 できごと 2.1 1800年代 2.2 1810年代 2.3 1820年代 2.4 1830年代 2.5 1840年代 2.6 1850年代 2.7 1860年代 2.8 1870年代 2.9 1880年代 2.10 1890年代 2.11 1900年代 3 文化 3.1 文学 3.2 音楽 3.3 思想 3.4 科学 3.5 技術 4 人物 4.1 ヨーロッパ 4.1.1 政治家・王族 4.1.1.1 フランス 4.1.1.2 オーストリア＝ハンガリー 4.1.1.3 ロシア 4.1.1.4 イギリス 4.1.1.5 ドイツ（プロイセンほかドイツ領邦を含む） 4.1.1.6 北欧 4.1.1.7 イタリア 4.1.1.8 スペイン 4.1.1.9 ベルギー 4.1.1.10 ギリシア 4.1.2 軍人 4.1.3 実業家 4.1.4 科学と技術 4.1.5 思想と哲学・人文諸学 4.1.6 宗教 4.1.7 文学 4.1.8 美術 4.1.9 音楽 4.1.10 社会事業家 4.1.11 探検家・旅行家 4.1.12 料...
Read more