## Recommended Posts

Posted (edited)

Hi,

I have two sorting programs. I want to concurrently execute them. I have written the following code using help from this forum:

        pid = os.getpid()  # current process
psutil.Process(pid).cpu_percent(interval=0)
th1.start()
th2.start()
while th1.is_alive():
print("CPU usage:" + str(psutil.Process(pid).cpu_percent(interval=1)))
while th2.is_alive():
print("CPU usage:" + str(psutil.Process(pid).cpu_percent(interval=1)))
print("all done")

Following is the output. This clearly shows that the execution of two threads is overlapping . Initially thread 1 starts prints the sorted list and then control goes to thread2 which prints first 4 numbers and then the control again goes to thread1 which prints the message " thread 1 ended". I want to run the thread 1 to completion and then to start the thread2. In Java we have "synchronize" command. Please guide me how to achieve mutual exclusion in Python.

The complete code is:

import random
import psutil
import os

class SSSort:
arr = []  # class var access by className.arr

def __init__(self, n):
self.n = n

def generate1000_5digit_RandomNum(self):
for i in range(self.n):
num = random.randint(10000, 99999)
SSSort.arr.append(num)

for j in range(self.n):
print("random {0}".format(SSSort.arr[j]))

def cpu_util_info(self):  # make sure different than posted
k = self.n
arr2 = SSSort.arr[:]

def find_the_min_element(k, i, arr2):
min = i
for j in range(i + 1, k):
if (arr2[j] < arr2[min]):
min = j
return min

def BubbleSort(k, arr2):
for i in arr2:
for j in range(len(arr2) - 1):
if (arr2[j] > arr2[j + 1]):
temp = arr2[j]
arr2[j] = arr2[j + 1]
arr2[j + 1] = temp

for i in enumerate(arr2):
print(i)

def find_the_min_element(k, i, arr2):
min = i
for j in range(i+1, k):
if (arr2[j] < arr2[min]):
min = j
return min

def selectionSort(k, arr2):
for i in range(self.n):
min = find_the_min_element(k, i, arr2)
# swap min and ith element of array
temp = arr2[min]
arr2[min]= arr2[i]
arr2[i] =temp

for i in enumerate(arr2):
print(i)

pid = os.getpid()  # current process
psutil.Process(pid).cpu_percent(interval=0)
th1.start()
th2.start()
while th1.is_alive():
print("CPU usage:" + str(psutil.Process(pid).cpu_percent(interval=1)))
while th2.is_alive():
print("CPU usage:" + str(psutil.Process(pid).cpu_percent(interval=1)))
print("all done")

def main(self):
self.generate1000_5digit_RandomNum()
self.cpu_util_info()

if __name__ == "__main__":
objSSSort = SSSort(10)
objSSSort.main()

Zulfi.

Quote

(0, 50735)
(1, 52575)
(2, 58431)
(3, 63366)
(4, 75071)
(5, 77648)
(6, 79961)
(7, 83715)
(8, 89010)
(9, 93857)
(0, 50735)
(1, 52575)
(2, 58431)
(3, 63366)
(4, 75071)
(5, 77648)
(6, 79961)
(7, 83715)
(8, 89010)
(9, 93857)
CPU usage:0.0
all done
memory usage=  15552512
disk usage=  sdiskusage(total=982900588544, used=141445500928, free=791455002624, percent=15.2)
virtual mem=  12050952192

Somebody please guide me, how to solve this problem.

Zulfi.

Edited by zak100

##### Share on other sites
1 hour ago, zak100 said:

Following is the output. This clearly shows that the execution of two threads is overlapping .

Having the threads do work in parallel is kind of the point having multiple threads.

1 hour ago, zak100 said:

I want to run the thread 1 to completion and then to start the thread2

Then make the program single threaded? Less room fo errors, easier to debug etc?
Anyway, a basic way to fulfil your requirements (two threads that does never run in parallel) is to delay the start of the second thread until the first thread is complete:

        th1 = threading.Thread(target=BubbleSort, args=(k, arr2))
th1.start()
while th1.is_alive():
print("CPU usage:" + str(psutil.Process(pid).cpu_percent(interval=1)))
th2.start()
while th2.is_alive():
print("CPU usage:" + str(psutil.Process(pid).cpu_percent(interval=1)))
print("thread2 ended")

1 hour ago, zak100 said:

Please guide me how to achieve mutual exclusion in Python.

I'm not sure this forum is the best place for me to write a long set of posts about multi threading required to cover this topic in enough detail, a blog would probably be better. Have you tried some other online sources or books?

##### Share on other sites
2 hours ago, zak100 said:

I want to run the thread 1 to completion and then to start the thread2. In Java we have "synchronize" command. Please guide me how to achieve mutual exclusion in Python.

e.g. make event object in the main thread, pass to child thread and set it at the end of execution. In the main thread wait till event is triggered.

54 minutes ago, Ghideon said:

Then make the program single threaded? Less room fo errors, easier to debug etc?

In majority of languages and OSes during writing app with GUI you should delay any long time (>50-100 miliseconds) taking operation to separate threads, and not block UI/the main thread, even if it is some basic stuff.

In Java on Android it is even forbidden to call e.g. network or database function/method on UI thread and OS immediately throws an exception.

##### Share on other sites
Posted (edited)

Hi,

Okay let's suppose we want to work on two case:

1) Concurrent Execution, may produce jumbled output

2) Running two threads accessing the data at the same time but their output is separate, i.e. we can avoid the jumbled output which I am seeing right now

Also for the both the above cases how can I get a combined CPU Usage ?

<Have you tried some other online sources or books? . >

Not in this case but for CPU usage I did a search. But I would do it now. Thanks.

Zulfi.

Edited by zak100

##### Share on other sites
1 minute ago, zak100 said:

How can I get a combined CPU Usage in the above case?

e.g. while loop checking whether any of two threads is alife (rather than just for one). Use logical or operator.

##### Share on other sites
15 minutes ago, Sensei said:

In majority of languages and OSes during writing app with GUI you should delay any long time (>50-100 miliseconds) taking operation to separate threads, and not block UI/the main thread, even if it is some basic stuff.

In Java on Android it is even forbidden to call e.g. network or database function/method on UI thread and OS immediately throws an exception.

I definitely agree. When multi threading and/or multi processing is required for a task, then I use it. I do prefer to use a simpler approach, such as a single thread, when that will achieve the same result.

2 hours ago, zak100 said:

In Java we have "synchronize" command.

In Java I prefer, when suitable, to use higher level libraries or packages such as java.util.concurrent https://docs.oracle.com/javase/8/docs/api/index.html?java/util/concurrent/package-summary.html

8 minutes ago, zak100 said:

How can I get a combined CPU Usage in the above case?

How do you define "combined"?

##### Share on other sites

Hi,

while th1.is_alive():
print("CPU usage:" + str(psutil.Process(pid).cpu_percent(interval=1)))
while th2.is_alive():
print("CPU usage:" + str(psutil.Process(pid).cpu_percent(interval=1)))

Quote

How do you define "combined"?

Right now its showing CPU usage for only one thread, I have to modify the above code to see for which code its printing CPU usage.

Yes, its possible to add the values of two CPU_percent(..) values.

Quote

e.g. while loop checking whether any of two threads is alife (rather than just for one). Use logical or operator.

Good idea, but first I want to see using large values whether its printing two cpu usage percentages or just one.

I would try and then get back to you people.

Zulfi.

##### Share on other sites
2 minutes ago, zak100 said:

Yes, its possible to add the values of two CPU_percent(..) values.

Can you provide something more rigorous?

Learning to express and/or understand (in plain english, using math, pseudocode or other suitable method) the requirements or needs is in my opinion central to solving programming tasks. I recommend using your questions as exercises.

##### Share on other sites
Posted (edited)
34 minutes ago, Ghideon said:

I definitely agree. When multi threading and/or multi processing is required for a task, then I use it. I do prefer to use a simpler approach, such as a single thread, when that will achieve the same result.

I prefer to have result fast.. As long as algorithm allows for easy migration to MT. Thanks to multithreading you can have results 8x/12x/.../48x/96x faster than on single thread.. depending how many cores/cpus are available.

Yet another level of making code harder to maintain and to read is migration to multi thousands GPU cores..

22 minutes ago, zak100 said:

Right now its showing CPU usage for only one thread, I have to modify the above code to see for which code its printing CPU usage.

Yes, its possible to add the values of two CPU_percent(..) values.

Edited by Sensei

##### Share on other sites
Posted (edited)

This might be interesting to you. CPU usage per thread.

Just use if thread th1 is alive then th1.get_cpu_percent(), render to string. Repeat with th2. Print final string.

Edited by Sensei

##### Share on other sites

Hi,

Sorry, I could not understand the code provided at:

Quote

This might be interesting to you. CPU usage per thread.

Just use if thread th1 is alive then th1.get_cpu_percent(), render to string. Repeat with th2. Print final string.

Instead I used a simple solution which i discussed earlier and I found the link also:

My output is this:

Quote

(141, 20322) t2
(142, 20335) t2
(143, 20348) t2
(144, 20425) t2
(145, 20426) t2
(146, 20477) t2
(147, 20511) t2
(148, 20587) t2
(149, 20613) t2
(150, 20849) t2
(151, 20855) t2
(152, 20919) t2
(153, 21241) t2
(0, 10174) t1
(1, 10293) t1
(2, 10467) t1
(3, 10591) t1
(4, 10614) t1
(5, 10700) t1
(6, 10707) t1
(7, 10718) t1
(8, 10943) t1
(9, 10953) t1
(10, 10990) t1
(11, 11026) t1
(12, 11053) t1
(13, 11098) t1
(14, 11209) t1
(15, 11346) t1
(16, 11378) t1
(17, 11405) t1
(18, 11420) t1
(154, 21314)

Though the output is still garbled but not I know which thread is printing what.

God bless you guys.

My problem is solved now.

Zulfi.

##### Share on other sites
14 hours ago, zak100 said:

My problem is solved now.

Good to hear that.

14 hours ago, zak100 said:

Instead I used a simple solution which i discussed earlier and I found the link also:

If you scroll down on that page you will find and intro to logging in Python. Generally logging* is an important part in any programming and useful, even in basic cases. Here is a quick example where thread name etc is printed in a consistent fashion by calling logging functions. Note the initial configuration that is done only once:

import logging

logging.basicConfig(level=logging.DEBUG,

logging.debug('start')
logging.debug('end')

2020-03-28 14:20:31,219 [DEBUG] (MainThread) start
2020-03-28 14:20:31,220 [DEBUG] (MainThread) end

Bold by me:

On 3/26/2020 at 9:41 PM, Sensei said:

I prefer to have result fast.. As long as algorithm allows for easy migration to MT. Thanks to multithreading you can have results 8x/12x/.../48x/96x faster than on single thread.. depending how many cores/cpus are available.

Yet another level of making code harder to maintain and to read is migration to multi thousands GPU cores..

Spot on. I think most of my experiences comes from being involved in projects where the bold part above did not apply or had to be weighed against other factors, hence my point of view above (default single thread, MT when required or a good option). Discussion in detail when to use MT vs other means of delivering fast results would be really interesting but unfortunately of topic

*) My opinion is strongly biased by my profession regarding this subject.

## Create an account

Register a new account