Dictionary infinite loop exiting unexpectedly












15















I was experimenting with various ways of creating an infinite loop in Python (other than the usual while True), and came up with this idea:



x = {0: None}



for i in x:

    del x[i]

    x[i+1] = None  # Value doesn't matter, so I set it to None

    print(i)


On paper, I traced out the way this would infinitely loop:




  1. I loop through the key's value in the dictionary

  2. I delete that entry.

  3. The current counter position in the loop + 1 will be the new key with value None which updates the dictionary.

  4. I output the current counter.


This, in my head, should output the natural numbers in a sort of infinite loop fashion:



0

1

2

3

4

5

.

.

.


I thought this idea was clever, however when I run it on Python 3.6, it outputs:



0

1

2

3

4


Yes, it somehow stopped after 5 iterations. Clearly, there is no base condition or sentinel value in the code block of the loop, so why is Python only running this code 5 times?






python dictionary infinite-loop







share|improve this question




















  • 4





    Mutating an object that you're iterating over is pretty much guaranteed to give you unpredictable results. In your specific case, I would guess that the 4 went into a hash bucket that had already been iterated over.

    – jasonharper
    8 hours ago






  • 1





    @jasonharper: CPython 3.6 is when they introduced the new compact dict implementation, so bucket locations wouldn't have this effect on 3.6. This looks like the dict needed a rebuild after the 5 was inserted, which would have re-compacted the compact array of entries and shifted the entry for the 5 to the left of the iterator position.

    – user2357112
    7 hours ago






  • 3





    You are making unwarranted assumptions about the dictionary iterator. The only assumption that you should make is that it isn't guaranteed to work if you add/remove from the dictionary while iterating over it. What actually ends up happening will depend on various implementation details.

    – juanpa.arrivillaga
    7 hours ago
















15















I was experimenting with various ways of creating an infinite loop in Python (other than the usual while True), and came up with this idea:



x = {0: None}



for i in x:

    del x[i]

    x[i+1] = None  # Value doesn't matter, so I set it to None

    print(i)


On paper, I traced out the way this would infinitely loop:




  1. I loop through the key's value in the dictionary

  2. I delete that entry.

  3. The current counter position in the loop + 1 will be the new key with value None which updates the dictionary.

  4. I output the current counter.


This, in my head, should output the natural numbers in a sort of infinite loop fashion:



0

1

2

3

4

5

.

.

.


I thought this idea was clever, however when I run it on Python 3.6, it outputs:



0

1

2

3

4


Yes, it somehow stopped after 5 iterations. Clearly, there is no base condition or sentinel value in the code block of the loop, so why is Python only running this code 5 times?






python dictionary infinite-loop







share|improve this question




















  • 4





    Mutating an object that you're iterating over is pretty much guaranteed to give you unpredictable results. In your specific case, I would guess that the 4 went into a hash bucket that had already been iterated over.

    – jasonharper
    8 hours ago






  • 1





    @jasonharper: CPython 3.6 is when they introduced the new compact dict implementation, so bucket locations wouldn't have this effect on 3.6. This looks like the dict needed a rebuild after the 5 was inserted, which would have re-compacted the compact array of entries and shifted the entry for the 5 to the left of the iterator position.

    – user2357112
    7 hours ago






  • 3





    You are making unwarranted assumptions about the dictionary iterator. The only assumption that you should make is that it isn't guaranteed to work if you add/remove from the dictionary while iterating over it. What actually ends up happening will depend on various implementation details.

    – juanpa.arrivillaga
    7 hours ago














15












15








15


2






I was experimenting with various ways of creating an infinite loop in Python (other than the usual while True), and came up with this idea:



x = {0: None}



for i in x:

    del x[i]

    x[i+1] = None  # Value doesn't matter, so I set it to None

    print(i)


On paper, I traced out the way this would infinitely loop:




  1. I loop through the key's value in the dictionary

  2. I delete that entry.

  3. The current counter position in the loop + 1 will be the new key with value None which updates the dictionary.

  4. I output the current counter.


This, in my head, should output the natural numbers in a sort of infinite loop fashion:



0

1

2

3

4

5

.

.

.


I thought this idea was clever, however when I run it on Python 3.6, it outputs:



0

1

2

3

4


Yes, it somehow stopped after 5 iterations. Clearly, there is no base condition or sentinel value in the code block of the loop, so why is Python only running this code 5 times?






python dictionary infinite-loop







share|improve this question
















I was experimenting with various ways of creating an infinite loop in Python (other than the usual while True), and came up with this idea:



x = {0: None}



for i in x:

    del x[i]

    x[i+1] = None  # Value doesn't matter, so I set it to None

    print(i)


On paper, I traced out the way this would infinitely loop:




  1. I loop through the key's value in the dictionary

  2. I delete that entry.

  3. The current counter position in the loop + 1 will be the new key with value None which updates the dictionary.

  4. I output the current counter.


This, in my head, should output the natural numbers in a sort of infinite loop fashion:



0

1

2

3

4

5

.

.

.


I thought this idea was clever, however when I run it on Python 3.6, it outputs:



0

1

2

3

4


Yes, it somehow stopped after 5 iterations. Clearly, there is no base condition or sentinel value in the code block of the loop, so why is Python only running this code 5 times?






python dictionary infinite-loop




python dictionary infinite-loop






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 7 hours ago









benvc

4,7071321




4,7071321










asked 8 hours ago









Suraj KothariSuraj Kothari

716417




716417








  • 4





    Mutating an object that you're iterating over is pretty much guaranteed to give you unpredictable results. In your specific case, I would guess that the 4 went into a hash bucket that had already been iterated over.

    – jasonharper
    8 hours ago






  • 1





    @jasonharper: CPython 3.6 is when they introduced the new compact dict implementation, so bucket locations wouldn't have this effect on 3.6. This looks like the dict needed a rebuild after the 5 was inserted, which would have re-compacted the compact array of entries and shifted the entry for the 5 to the left of the iterator position.

    – user2357112
    7 hours ago






  • 3





    You are making unwarranted assumptions about the dictionary iterator. The only assumption that you should make is that it isn't guaranteed to work if you add/remove from the dictionary while iterating over it. What actually ends up happening will depend on various implementation details.

    – juanpa.arrivillaga
    7 hours ago














  • 4





    Mutating an object that you're iterating over is pretty much guaranteed to give you unpredictable results. In your specific case, I would guess that the 4 went into a hash bucket that had already been iterated over.

    – jasonharper
    8 hours ago






  • 1





    @jasonharper: CPython 3.6 is when they introduced the new compact dict implementation, so bucket locations wouldn't have this effect on 3.6. This looks like the dict needed a rebuild after the 5 was inserted, which would have re-compacted the compact array of entries and shifted the entry for the 5 to the left of the iterator position.

    – user2357112
    7 hours ago






  • 3





    You are making unwarranted assumptions about the dictionary iterator. The only assumption that you should make is that it isn't guaranteed to work if you add/remove from the dictionary while iterating over it. What actually ends up happening will depend on various implementation details.

    – juanpa.arrivillaga
    7 hours ago








4




4





Mutating an object that you're iterating over is pretty much guaranteed to give you unpredictable results. In your specific case, I would guess that the 4 went into a hash bucket that had already been iterated over.

– jasonharper
8 hours ago





Mutating an object that you're iterating over is pretty much guaranteed to give you unpredictable results. In your specific case, I would guess that the 4 went into a hash bucket that had already been iterated over.

– jasonharper
8 hours ago




1




1





@jasonharper: CPython 3.6 is when they introduced the new compact dict implementation, so bucket locations wouldn't have this effect on 3.6. This looks like the dict needed a rebuild after the 5 was inserted, which would have re-compacted the compact array of entries and shifted the entry for the 5 to the left of the iterator position.

– user2357112
7 hours ago





@jasonharper: CPython 3.6 is when they introduced the new compact dict implementation, so bucket locations wouldn't have this effect on 3.6. This looks like the dict needed a rebuild after the 5 was inserted, which would have re-compacted the compact array of entries and shifted the entry for the 5 to the left of the iterator position.

– user2357112
7 hours ago




3




3





You are making unwarranted assumptions about the dictionary iterator. The only assumption that you should make is that it isn't guaranteed to work if you add/remove from the dictionary while iterating over it. What actually ends up happening will depend on various implementation details.

– juanpa.arrivillaga
7 hours ago





You are making unwarranted assumptions about the dictionary iterator. The only assumption that you should make is that it isn't guaranteed to work if you add/remove from the dictionary while iterating over it. What actually ends up happening will depend on various implementation details.

– juanpa.arrivillaga
7 hours ago












2 Answers
2






active

oldest

votes


















19














There is no guarantee that you will iterate over all your dict entries if you mutate it in your loop. From the docs:




Iterating views while adding or deleting entries in the dictionary may
raise a RuntimeError or fail to iterate over all entries.




You could create an "enumerated" infinite loop similar to your initial attempt using itertools.count(). For example:



from itertools import count



for i in count():

    print(i)

    # don't run this without some mechanism to break the loop, i.e.

    # if (i == 10):

    #     break



# OUTPUT

# 0

# 1

# 2

# ...and so on





share|improve this answer


























  • Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

    – Suraj Kothari
    7 hours ago






  • 1





    @SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

    – benvc
    7 hours ago





















4














I just tested your code in python2 and python3



python3 output

0,1,2,3,4

python2

0,1,2,3,4,5,6,7


One thing comes to mind that could be going on. Either there is only a certain amount of memory being allocated in your dictionary when you create the first key value and when you delete the key value we do not allocate any memory or deallocate the memory you are just removing the value. Once all the allocated memory is used it exits. Because if you run without that del you will get this error



RuntimeError: dictionary changed size during iteration


So python creates enough memory for that key value and a few more and once it is used up theres no more memory allocated for your dictionary.






share|improve this answer


























  • Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

    – Suraj Kothari
    8 hours ago






  • 1





    I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

    – Suraj Kothari
    7 hours ago











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54316557%2fdictionary-infinite-loop-exiting-unexpectedly%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























2 Answers
2






active

oldest

votes








2 Answers
2






active

oldest

votes









active

oldest

votes






active

oldest

votes









19














There is no guarantee that you will iterate over all your dict entries if you mutate it in your loop. From the docs:




Iterating views while adding or deleting entries in the dictionary may
raise a RuntimeError or fail to iterate over all entries.




You could create an "enumerated" infinite loop similar to your initial attempt using itertools.count(). For example:



from itertools import count



for i in count():

    print(i)

    # don't run this without some mechanism to break the loop, i.e.

    # if (i == 10):

    #     break



# OUTPUT

# 0

# 1

# 2

# ...and so on





share|improve this answer


























  • Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

    – Suraj Kothari
    7 hours ago






  • 1





    @SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

    – benvc
    7 hours ago


















19














There is no guarantee that you will iterate over all your dict entries if you mutate it in your loop. From the docs:




Iterating views while adding or deleting entries in the dictionary may
raise a RuntimeError or fail to iterate over all entries.




You could create an "enumerated" infinite loop similar to your initial attempt using itertools.count(). For example:



from itertools import count



for i in count():

    print(i)

    # don't run this without some mechanism to break the loop, i.e.

    # if (i == 10):

    #     break



# OUTPUT

# 0

# 1

# 2

# ...and so on





share|improve this answer


























  • Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

    – Suraj Kothari
    7 hours ago






  • 1





    @SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

    – benvc
    7 hours ago
















19












19








19







There is no guarantee that you will iterate over all your dict entries if you mutate it in your loop. From the docs:




Iterating views while adding or deleting entries in the dictionary may
raise a RuntimeError or fail to iterate over all entries.




You could create an "enumerated" infinite loop similar to your initial attempt using itertools.count(). For example:



from itertools import count



for i in count():

    print(i)

    # don't run this without some mechanism to break the loop, i.e.

    # if (i == 10):

    #     break



# OUTPUT

# 0

# 1

# 2

# ...and so on





share|improve this answer















There is no guarantee that you will iterate over all your dict entries if you mutate it in your loop. From the docs:




Iterating views while adding or deleting entries in the dictionary may
raise a RuntimeError or fail to iterate over all entries.




You could create an "enumerated" infinite loop similar to your initial attempt using itertools.count(). For example:



from itertools import count



for i in count():

    print(i)

    # don't run this without some mechanism to break the loop, i.e.

    # if (i == 10):

    #     break



# OUTPUT

# 0

# 1

# 2

# ...and so on






share|improve this answer














share|improve this answer



share|improve this answer








edited 7 hours ago

























answered 8 hours ago









benvcbenvc

4,7071321




4,7071321













  • Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

    – Suraj Kothari
    7 hours ago






  • 1





    @SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

    – benvc
    7 hours ago





















  • Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

    – Suraj Kothari
    7 hours ago






  • 1





    @SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

    – benvc
    7 hours ago



















Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

– Suraj Kothari
7 hours ago





Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

– Suraj Kothari
7 hours ago




1




1





@SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

– benvc
7 hours ago







@SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

– benvc
7 hours ago















4














I just tested your code in python2 and python3



python3 output

0,1,2,3,4

python2

0,1,2,3,4,5,6,7


One thing comes to mind that could be going on. Either there is only a certain amount of memory being allocated in your dictionary when you create the first key value and when you delete the key value we do not allocate any memory or deallocate the memory you are just removing the value. Once all the allocated memory is used it exits. Because if you run without that del you will get this error



RuntimeError: dictionary changed size during iteration


So python creates enough memory for that key value and a few more and once it is used up theres no more memory allocated for your dictionary.






share|improve this answer


























  • Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

    – Suraj Kothari
    8 hours ago






  • 1





    I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

    – Suraj Kothari
    7 hours ago
















4














I just tested your code in python2 and python3



python3 output

0,1,2,3,4

python2

0,1,2,3,4,5,6,7


One thing comes to mind that could be going on. Either there is only a certain amount of memory being allocated in your dictionary when you create the first key value and when you delete the key value we do not allocate any memory or deallocate the memory you are just removing the value. Once all the allocated memory is used it exits. Because if you run without that del you will get this error



RuntimeError: dictionary changed size during iteration


So python creates enough memory for that key value and a few more and once it is used up theres no more memory allocated for your dictionary.






share|improve this answer


























  • Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

    – Suraj Kothari
    8 hours ago






  • 1





    I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

    – Suraj Kothari
    7 hours ago














4












4








4







I just tested your code in python2 and python3



python3 output

0,1,2,3,4

python2

0,1,2,3,4,5,6,7


One thing comes to mind that could be going on. Either there is only a certain amount of memory being allocated in your dictionary when you create the first key value and when you delete the key value we do not allocate any memory or deallocate the memory you are just removing the value. Once all the allocated memory is used it exits. Because if you run without that del you will get this error



RuntimeError: dictionary changed size during iteration


So python creates enough memory for that key value and a few more and once it is used up theres no more memory allocated for your dictionary.






share|improve this answer















I just tested your code in python2 and python3



python3 output

0,1,2,3,4

python2

0,1,2,3,4,5,6,7


One thing comes to mind that could be going on. Either there is only a certain amount of memory being allocated in your dictionary when you create the first key value and when you delete the key value we do not allocate any memory or deallocate the memory you are just removing the value. Once all the allocated memory is used it exits. Because if you run without that del you will get this error



RuntimeError: dictionary changed size during iteration


So python creates enough memory for that key value and a few more and once it is used up theres no more memory allocated for your dictionary.







share|improve this answer














share|improve this answer



share|improve this answer








edited 7 hours ago

























answered 8 hours ago









turtlesalldayturtlesallday

281112




281112













  • Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

    – Suraj Kothari
    8 hours ago






  • 1





    I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

    – Suraj Kothari
    7 hours ago



















  • Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

    – Suraj Kothari
    8 hours ago






  • 1





    I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

    – Suraj Kothari
    7 hours ago

















Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

– Suraj Kothari
8 hours ago





Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

– Suraj Kothari
8 hours ago




1




1





I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

– Suraj Kothari
7 hours ago





I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

– Suraj Kothari
7 hours ago


















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid …



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54316557%2fdictionary-infinite-loop-exiting-unexpectedly%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







-

Popular posts from this blog

How to reconfigure Docker Trusted Registry 2.x.x to use CEPH FS mount instead of NFS and other traditional...

Image
0 I am having trouble trying to reconfigure my DTR container to try and use a CEPH FS mount we created to act as a storage backend for our image repository. The Docs are very vague on this and nothing exactly explains on how to do it. Has anyone yet connected the dots? linux filesystems mount storage docker share | improve this question asked Feb 1 at 6:40 jRapp3r jRapp3r 1 1 add a comment  | 
Read more

is 'sed' thread safe

Image
2 If I have a shell/python script that uses sed to modify a file in place based on user inputs, and then two users run the same script at the same time or approx. same time, is 'sed' thread safe ? Or perhaps it is not an issue because the file_descripor that was opened by the first thread will be used to lock the file anyway ? thx linux sed python share | improve this question edited 7 hours ago Jeff Schaller 42.9k 11 59 137 asked 7 hours ago terreys terreys
Read more

How to make a Squid Proxy server?

Image
1 I want to set up squid proxy server for monitoring user activity in a network and Internet Web URL filtering for user. I also want to configure the user login, user log, user access group, and block unwanted web sites in my network. Please help me and guide me the full process to do this. server proxy squid share | improve this question edited Jul 31 '12 at 14:45 Jorge Castro 37.1k 107 422 617 asked Jul 31 '12 at 5:16 Goivind Tiwari Goivind Tiwari
Read more