NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the ‘ssl’ module is compiled with ‘LibreSSL 2.8.3’.

site-packages/urllib3/__init__.py:34: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020

Turns out that urllib3 version 2 wants OpenSSL to work properly. The earlier versions of urllib3 version 2 even stopped entirely, the later ones throw this warning. But my current ssl module appears to be LibreSSL. The idea is to install an urllib3 that is compatible with LibreSSL.

Solution

pip install urllib3==1.26.20

An older version of urllib3 series 1 works, I picked the latest. Now my app runs without the warning.

How to install the yaml package for Python?

I want to read the config for my backend from a yaml file and when installing the yaml package I am getting the following error:

pip install yaml
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
ERROR: Could not find a version that satisfies the requirement yaml (from versions: none)
ERROR: No matching distribution found for yaml

Quick Answer

pip install pyyaml

More Details

This article explaining how to read yaml config files in Python doesn’t show how to install the package, which should be straightforward, but there is no such package as yaml. A quick search on pypi.org found this package: https://pypi.org/project/PyYAML/

I tried it and it works like a charm with the same syntax as in the blog post. Now you can go ahead and try Ram’s example.

Fix: No GPU support in Tensorflow

I came across a problem where my Tensorflow installation did not recognize the installed gpu, despite of Cuda and Nvidia drivers being installed properly.

test:

python3 -c “import tensorflow as tf; print(tf.config.list_physical_devices(‘GPU’))”

returned an empty list. Furthermore, it tells it cannot find the cuda library:

2024-01-30 14:57:42.015454: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.

Output of the Nvidia tool is correct and shows Cuda is installed:

nvidia-smi

ubuntu@ip-bla-foo:~/build-nb$  nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Mon_Apr__3_17:16:06_PDT_2023
Cuda compilation tools, release 12.1, V12.1.105
Build cuda_12.1.r12.1/compiler.32688072_0

Which tells us it is version 12. Ahhh!💡

Now, 12 is a version from 2023 and my idea was that Tensorflow 2.13 might not know this version, see https://blog.tensorflow.org/2023/11/whats-new-in-tensorflow-2-15.html

Ok, the latest version pip offered was TF 2.13 on Python 3.8. Here is the fix:

  1. upgrade Python: sudo apt install python3.9
  2. a new venv: virtualenv –python /usr/bin/python3.9 ~/.env-python3.9
  3. source ~/.env-python3.9/bin/activate
  4. pip install –upgrade pip
  5. python3 -m pip install tensorflow[and-cuda]==2.15.0.post1

Test: python3 -c “import tensorflow as tf; print(tf.config.list_physical_devices(‘GPU’))”

2024-01-30 15:27:04.458720: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-01-30 15:27:04.458772: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-01-30 15:27:04.459601: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-01-30 15:27:04.465334: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-01-30 15:27:05.115551: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-01-30 15:27:05.560865: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2024-01-30 15:27:05.585883: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2024-01-30 15:27:05.586100: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

Now we see the GPU in Tensorflow.

Qt application on Windows not starting

When you click on an applications exe file and nothing happens, you might wonder what it does (if anything) and find a way to debug what is happening. By default, the binary does not output any log to the console. But there is a way to do that.

You can rebuild your Qt app using the console qmake flag in order to get some useful debugging output on the console.

CONFIG += console

either in your QtCreator Build Settings or as a command line argument to qmake. Then run the app exe from a Windows terminal such as PowerShell and you will see the same output you would see in QtCreator, including your qDebug outputs. You can also click on it in the explorer and it opens a standard terminal showing the console output.

This can be very useful, e.g. when debugging a sandbox created with windeployqt.

Printing Stack Traces in Python Exceptions: A Comprehensive Guide

Uncover the Root of Errors with Detailed Tracebacks

Encountering errors in Python is inevitable. But with stack traces, you can pinpoint the exact location of an exception and diagnose issues effectively. Here’s a step-by-step guide:

1. Import the traceback Module:

import traceback

2. Utilize Exception Handling:

try:
processEvent() # Call the function that might raise an exception
except Exception as e:
print("Error encountered:", e)
traceback.print_exc() # Print the detailed stack trace

Key Points:

  • traceback.print_exc(): Prints a formatted traceback to the console.
  • Error Message: The print("Error encountered:", e) line displays the error message itself.
  • Traceback Structure:
    • Each line represents a function call leading to the exception.
    • The topmost line indicates the most recent call, and subsequent lines trace the sequence backward.

Example Output:

Error encountered: An error occurred!
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 2, in processEvent
Exception: An error occurred!

Leave a comment, or reach out if you have a question.

Additional Insights:

  • Customizing Output:
    • traceback.format_exc() returns the formatted traceback as a string for further manipulation.
    • traceback.print_exception() offers more control over output formatting.
  • Logging Stack Traces: Consider logging stack traces for debugging or analysis purposes.

Troubleshooting with Confidence:

  • By effectively printing and understanding stack traces, you’ll be equipped to resolve Python exceptions efficiently and maintain code stability.

root symlink on MacOS BigSur

After upgrading my Macbook to BigSur (I skipped Catalina) my MongoDB didn’t want to start anymore. A quick inspection showed that the /data/db folder I used to have to house my db files didn’t exist anymore. BigSur has removed it 😱 but luckily I found a backup.

It appears that the root / folder on MacOS is now readonly, and I wasn’t able to copy the backup to its original place. I chose the Mac disk instead, and it now sits in /System/Volumes/Data/data

What was left is to create a symlink to /data, and the way to do that is by using the /etc/synthetic.conf file:

data    /System/Volumes/Data/data

This fixes the /data folder I used to have for MongoDB.

bindService() not working on Android SDK 30 and 31

This can be fixed by adding the following code snippet to the Android Manifest:

<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.example.app">
  <queries>
    <package android:name="com.example.service" />
  </queries>

  <application>
    ...
  </application>
</manifest>

In SDK 29 it was working, but in SDK 30 and 31 bindService() returned always false and did not throw any exception. The system simply couldn’t find the service, even though the service was exported using android:exported="true".

Work with large numbers of files in a folder

A few shell commands to list, sort or move files in a large folder.

I have a component that writes a log file every time a task is executed, and they all go to the same folder. ls or Midnight Commander take a long time on that folder. I’d like to move the old files out of the way, e.g. to a folder called, well, old. Or I want to delete them. Also, I’d like to list the most recent files, maybe the largest of today, or just the last file that was written. Here a few commands that work on my Ubuntu bash:

find /data/logs/ -maxdepth 1 -mtime +30 -type f -exec mv "{}" old/ \;

Finds the files older than 30 days and moves them to a folder old/

find /data/logs/ -maxdepth 1 -type f -mtime -1

Lists the files no more than 1 day old.

find /data/logs/ -maxdepth 1 -type f -mtime -1 -exec du -ah  "{}" \;  | sort -n -r | head -n 10

Finds the files no older than 1 day, sorts them by size, and shows the largest 10.

cmake Problem: pthread and iOS

A cmake build issue I came across that occurs when building with cmake and the latest version of XCode. The macro find_package(Threads) fails with

-- Looking for pthread.h - not found

Could NOT find Threads (missing: Threads_FOUND)

Digging into the log file tells it is having problems with code signing. What happens is that try_compile() compiles a very small program using pthread, and even this tiny piece needs to be code signed:

Code Signing Error: Code signing is required for product type ‘Application’ in SDK ‘iOS 13.2’

After playing around a bit, I found this workaround:

set(CMAKE_TRY_COMPILE_PLATFORM_VARIABLES "CMAKE_XCODE_ATTRIBUTE_CODE_SIGNING_ALLOWED")
set(CMAKE_XCODE_ATTRIBUTE_CODE_SIGNING_ALLOWED "NO")

find_package(Threads REQUIRED)

unset(CMAKE_TRY_COMPILE_PLATFORM_VARIABLES)
unset(CMAKE_XCODE_ATTRIBUTE_CODE_SIGNING_REQUIRED)

Now it is able to compile the little test program and configures threads properly in cmake.