# Getting stacktraces on crashes

The Compiler
`<mail@qutebrowser.org>`

When there is a fatal crash in qutebrowser - most of the times a segfault - the crash report usually doesn’t contain much viable information, as these crashes usually happen inside of the Qt mainloop in C++.

To know what the issue is, a stack trace with debugging symbols is required.

The rest of this guide is quite Linux specific, though there is a section for Windows at the end.

## Crashes which can be reproduced

If a crash can be reproduced, packages with debugging symbols should be installed, and the crash should be reproduced under gdb.

### Getting debugging symbols

#### Debian/Ubuntu/…

For Debian based systems (Debian, Ubuntu, Linux Mint, …), debug information is for QtWebEngine is available in a dedicated repository. Enable that repository (Debian, Ubuntu, Linux Mint) and install the debug packages:

    # apt install python3-dbg python3-pyqt5-dbg python3-pyqt5.qtwebengine-dbg libqt5webengine5-dbgsym libqt5webenginecore5-dbgsym

or with the QtWebKit backend:

    # apt install python3-dbg python3-pyqt5-dbg python3-pyqt5.qtwebkit-dbg libqt5webkit5-dbg

#### Fedora

For Fedora you first need to install the dnf/yum-utils:

    # dnf install dnf-utils

Or:

    # yum install yum-utils

Then install the needed debuginfo packages:

    # debuginfo-install python3 qt5-qtwebengine python3-qt5-webengine python3-qt5-base python-qt5 python3-qt5 python3-qt5-webkit

#### Archlinux

For Archlinux, no debug information is provided. You can either compile Qt yourself (which will take a few hours even on a modern machine) or use debugging symbols compiled/packaged by me (x86_64 only).

##### To install my pre-built packages

First download and sign the key:

    # pacman-key -r 0xD6A1C70FE80A0C82
    $ pacman-key -f 0xD6A1C70FE80A0C82
      Key fingerprint = 14AF EC28 70C6 4863 C5C7  ACCB D6A1 C70F E80A 0C82
    # pacman-key --lsign-key 0xD6A1C70FE80A0C82

Then edit your `/etc/pacman.conf` to add the repository to the bottom:

    [qt-debug]
    Server = https://qutebrowser.org/qt-debug/$arch

Then install the packages:

    # pacman -Suy python-pyqt5-debug qt5-base-debug qt5-webkit-debug qt5-webengine-debug

The `-debug` packages conflict with the non-debug variants - it’s safe to remove them.

##### To compile by yourself

Note that building Qt will likely take multiple hours, even on a recent system. I’d also expect it to take around 6 GB of RAM and 30 GB of disk space for a successful compile run.

    $ git clone https://github.com/qutebrowser/qt-debug-pkgbuild.git
    $ cd qt-debug-pkgbuild
    $ export DEBUG_CFLAGS='-ggdb3 -fvar-tracking-assignments -Og'
    $ export DEBUG_CXXFLAGS='-ggdb3 -fvar-tracking-assignments -Og'
    $ cd qt5
    $ makepkg -si --pkg qt5-base-debug,qt5-webkit-debug,qt5-webengine-debug
    $ cd ../pyqt5
    $ makepkg -si --pkg python-pyqt5-debug

### Getting the stack trace

First install `gdb` on your system if it’s not installed already.

Then run qutebrowser directly inside gdb like this:

    $ gdb -ex r --args $(readlink -f $(which python3)) -m qutebrowser --debug --temp-basedir

Note qutebrowser/gdb will take a long time to start. After you reproduce the crash, you should now see something like:

    Program received signal SIGSEGV, Segmentation fault.
    ...
    (gdb)

Now enter these commands at the gdb prompt:

    (gdb) set pagination off
    (gdb) set logging overwrite on
    (gdb) set logging on
    (gdb) bt
    (gdb) quit

This will create a `gdb.txt` in your current directory.

Copy the last few lines of the debug log (before you got the gdb prompt) and the full content of `gdb.txt` into the bug report. Please also add some words about what you were doing (or what pages you visited) before the crash happened.

## Crashes which can NOT be reproduced

If you cannot reproduce the problem, you need to check if a coredump got written somewhere. You should not install debug symbols as they won’t match the generated coredump.

First install `gdb` on your system if it’s not installed already.

Then check the file `/proc/sys/kernel/core_pattern` on your system. If it does not start with a `|` character (pipe), check if there is a file named `core` or `core.NNNN` in the directory from that file, or in the current directory.

If so, execute gdb like this:

    $ gdb $(readlink -f $(which python3)) /path/to/core

If your `/proc/sys/kernel/core_pattern` contains something like `|/usr/lib/systemd/systemd-coredump`, use `coredumpctl` to run gdb:

    $ coredumpctl gdb $(readlink -f $(which python3))

### Getting the stack trace

Now enter these commands at the gdb prompt:

    (gdb) set pagination off
    (gdb) set logging overwrite on
    (gdb) set logging on
    (gdb) bt
    (gdb) quit

Copy the content of `gdb.txt` into the bug report. Please also add some words about what you were doing (or what pages you visited) before the crash happened.

## For Windows

First install DebugDiag from Microsoft.

If you see the qutebrowser.exe has stopped working window, do not click "Close the program". Instead, open your task manager, there right-click on `qutebrowser.exe` and select "Create dump file". Remember the path of the dump file displayed there.

If you do not see such a window, instead run DebugDiag 2 Collection while qutebrowser is still running. There, use Add Rule → Crash → A specific process and select `qutebrowser.exe`. Accept the Advanced Configuration as-is and select a location to save dump files. Finally, tell DebugDiag to activate the rule and reproduce the crash. After a while, a log file (`.txt`) and crash dump should appear in that directory.

Finally, run the DebugDiag 2 Analysis tool. There, check CrashHangAnalysis and add your crash dump via Add Data files. Then click Start analysis.

Close the Internet Explorer which opens when it’s done and use the folder-button at the top left to get to the reports. There, find the report file (as well as the logfile, if any), zip them (important, as some mail providers like GMail corrupt the file otherwise) and send them to mail@qutebrowser.org.