[Bug 1767943] [NEW] [Ubuntu 18.04][sosreport] caught exception in sosreport at plugin method "logs.collect()"

[bugproxy]

Public bug reported:

== Comment: #0 - ABDUL HALEEM <***@in.ibm.com> - 2018-03-16 01:59:41 ==
Problem:
------------
While running sosreport on ubuntu 18.04 powerpc machine it got exception when file is large

Running 27/60: logs... caught exception in plugin method "logs.collect()"
writing traceback to sos_logs/logs-plugin-errors.txt
Running 30/60: lxd...

[plugin:lxd] command 'lxc list' timed out after 300s
[plugin:lxd] command 'lxc profile list' timed out after 300s


Machine : Witherspoon bare-metal DD2.2
sosreport 3.5-1ubuntu2

uname -a
Linux ltciofvtr-spoon1 4.15.0-12-generic #13-Ubuntu SMP Wed Mar 7 21:37:03 UTC 2018 ppc64le
ppc64le ppc64le GNU/Linux

# free -h
total used free shared buff/cache available
Mem: 503G 7.1G 493G 21M 2.4G 493G
Swap: 2.0G 0B 2.0G

- - - steps to reproduce - - -
sosreport

sosreport (version 3.5)

This command will collect system configuration and diagnostic
information from this Ubuntu system. An archive containing the collected
information will be generated in /tmp/sos.2_u6n8p3.

For more information on Ubuntu visit:

http://www.ubuntu.com/

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.

No changes will be made to system configuration.

Press ENTER to continue, or CTRL-C to quit.

Please enter your first initial and last name [ltciofvtr-spoon1]:
Please enter the case id that you are generating this report for []:

Setting up archive ...
Setting up plugins ...

Running plugins. Please wait ...

Running 27/60: logs... caught exception in plugin method "logs.collect()"
writing traceback to sos_logs/logs-plugin-errors.txt
Running 30/60: lxd...

[plugin:lxd] command 'lxc list' timed out after 300s
[plugin:lxd] command 'lxc profile list' timed out after 300s
[plugin:lxd] command 'lxc image list' timed out after 300s
Running 60/60: xfs...
Creating compressed archive...

Your sosreport has been generated and saved in:
/tmp/sosreport-ltciofvtr-spoon1-20180316023240.tar.xz

The checksum is: 8a2d4464d0ee94a306f6529a6a92f4e6

Please send this file to your support representative.

== Comment: #6 - ABDUL HALEEM <***@in.ibm.com> - 2018-04-27 00:18:45 ==
Machine is available only for today for debugging and I am still able to reproduce this bug on 4.15.0-17-generic

Please mirror this as soon as possible.


***@ltciofvtr-spoon1:~# sosreport

sosreport (version 3.5)

This command will collect system configuration and diagnostic
information from this Ubuntu system. An archive containing the collected
information will be generated in /tmp/sos.7debi_it.

For more information on Ubuntu visit:

http://www.ubuntu.com/

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.

No changes will be made to system configuration.

Press ENTER to continue, or CTRL-C to quit.

Please enter your first initial and last name [ltciofvtr-spoon1]:
Please enter the case id that you are generating this report for []:

Setting up archive ...
Setting up plugins ...
Running plugins. Please wait ...

Running 29/63: logs... caught exception in plugin method "logs.collect()"
writing traceback to sos_logs/logs-plugin-errors.txt
Running 57/63: systemd...

== Comment: #8 - Urvashi Jawere <***@in.ibm.com> - 2018-04-27 06:46:57 ==
Hi Abdul,

I tried with a small fix and I was able to create the sosreport. Python 3 doesn't have decode anymore.
Attaching the patch fix with the bug.

== Comment: #9 - Urvashi Jawere <***@in.ibm.com> - 2018-04-27
06:47:33 ==


== Comment: #10 - Urvashi Jawere <***@in.ibm.com> - 2018-04-27 06:50:53 ==
After fix, I was able to collect sosreport from machine-

***@ltciofvtr-spoon1:/usr/share/sosreport/sos/plugins# sosreport

sosreport (version 3.5)

This command will collect system configuration and diagnostic
information from this Ubuntu system. An archive containing the collected
information will be generated in /tmp/sos.4i288ell.

For more information on Ubuntu visit:

http://www.ubuntu.com/

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.

No changes will be made to system configuration.

Press ENTER to continue, or CTRL-C to quit.

Please enter your first initial and last name [ltciofvtr-spoon1]: try
Please enter the case id that you are generating this report for []: bugfix

Setting up archive ...
Setting up plugins ...
Running plugins. Please wait ...

Running 63/63: xfs...
Creating compressed archive...

Your sosreport has been generated and saved in:
/tmp/sosreport-try.bugfix-20180427071050.tar.xz

The checksum is: 03df61167b547a242ce3f18b588f3c47

Please send this file to your support representative.

***@ltciofvtr-spoon1:/usr/share/sosreport/sos/plugins# ls

== Comment: #11 - Urvashi Jawere <***@in.ibm.com> - 2018-04-30
01:32:55 ==

** Affects: sosreport (Ubuntu)
Importance: Undecided
Assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
Status: New


** Tags: architecture-ppc64le bugnameltc-165778 severity-high targetmilestone-inin---

** Tags added: architecture-ppc64le bugnameltc-165778 severity-high
targetmilestone-inin---
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/sosreport/+bug/1767943/+subscriptions

[bugproxy]

Default Comment by Bridge

** Attachment added: "logs-plugin-errors"
https://bugs.launchpad.net/bugs/1767943/+attachment/5130521/+files/logs-plugin-errors.txt
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/sosreport/+bug/1767943/+subscriptions

[bugproxy]

Default Comment by Bridge

** Attachment added: "Sosreport_final_Patch"
https://bugs.launchpad.net/bugs/1767943/+attachment/5130522/+files/sosreport.patch

** Changed in: ubuntu
Assignee: (unassigned) => Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)

** Package changed: ubuntu => sosreport (Ubuntu)
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/sosreport/+bug/1767943/+subscriptions

[Frank Heimes]

** Also affects: ubuntu-power-systems
Importance: Undecided
Status: New

** Changed in: ubuntu-power-systems
Importance: Undecided => High

** Changed in: ubuntu-power-systems
Assignee: (unassigned) => Canonical Server Team (canonical-server)

** Tags added: triage-g
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1767943/+subscriptions

[Frank Heimes]

** Changed in: ubuntu-power-systems
Status: New => Triaged

** Changed in: ubuntu-power-systems
Assignee: Canonical Server Team (canonical-server) => David Britton (davidpbritton)
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1767943/+subscriptions

[Eric Desrochers]

*** This bug is a duplicate of bug 1761442 ***
https://bugs.launchpad.net/bugs/1761442

A colleague of mine (the one who own the original LP and upstream bug)
and myself will upload the change in Ubuntu as soon as it got release
upstream.

- Eric
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1767943/+subscriptions

[Eric Desrochers]

*** This bug is a duplicate of bug 1761442 ***
https://bugs.launchpad.net/bugs/1761442

I found out that the bug already exist LP[1] and Upstream[2]


[1] - Launchpad:
-------------------------------
https://bugs.launchpad.net/ubuntu/+source/sosreport/+bug/1761442
-------------------------------

[2] - Upstream:
-------------------------------
https://github.com/sosreport/sos/pull/1267
https://github.com/sosreport/sos/pull/1267/commits/bf8433945aeced7c0d73686ec83d826cc33bbd28

Looks like a duplicate of #1267 - fix is in the review queue, should
land next week with any luck.

We're planning to make a 3.5.1 interim release shortly to help distro packagers who cannot wait until 3.6 final is out - this fix will be included in that tag.
-------------------------------

** This bug has been marked a duplicate of bug 1761442
sosreport: AttributeError: 'str' object has no attribute 'decode'
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1767943/+subscriptions

[Eric Desrochers]

I will handle the upstream work on your behalf.

I have reported the situation upstream:
https://github.com/sosreport/sos/issues/1301

- Eric

** Bug watch added: github.com/sosreport/sos/issues #1301
https://github.com/sosreport/sos/issues/1301
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1767943/+subscriptions

[Eric Desrochers]

It passes just fine for me if using "--all-logs" with the same
reproducer.

$ sosreport --all-logs
Running 27/57: logs...
Running 28/57: lsbrelease...
Running 29/57: lvm2...
Running 30/57: lxd...
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1767943/+subscriptions

[Eric Desrochers]

Can you also reproduce using the following argument :

--all-logs
Tell plugins to collect all possible log data ignoring any size limits and including logs in non-default locations. This option may significantly increase the size of reports.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1767943/+subscriptions

[Eric Desrochers]

Did you consider submitting your proposal patch to sosreport upstream
[https://github.com/sosreport/sos] to get the credit for it ?

Once merged upstream, I can submit it to debian and then upload into
Ubuntu (cosmic and stable) in that same order.

Regards,
Eric
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1767943/+subscriptions

[Eric Desrochers]

Reproducible (same behaviour) on amd64 ARCH as well ^

caught exception in plugin method "logs.collect()"
writing traceback to sos_logs/logs-plugin-errors.txt
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1767943/+subscriptions

[Eric Desrochers]

I was able to reproduce on powerpc ARCH by creating a fake "kern.log" of
1GB size.

Running 27/57: logs...
caught exception in plugin method "logs.collect()"
writing traceback to sos_logs/logs-plugin-errors.txt

# sos_logs/logs-plugin-errors.txt
Traceback (most recent call last):
File "/usr/share/sosreport/sos/sosreport.py", line 1300, in collect
plug.collect()
File "/usr/share/sosreport/sos/plugins/__init__.py", line 877, in collect
self._collect_strings()
File "/usr/share/sosreport/sos/plugins/__init__.py", line 860, in _collect_strings
(content.splitlines()[0]).decode('utf8', 'ignore'))
AttributeError: 'str' object has no attribute 'decode'
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1767943/+subscriptions

[Eric Desrochers]

** Changed in: sosreport (Ubuntu)
Status: New => In Progress
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1767943

Title:
[Ubuntu 18.04][sosreport] caught exception in sosreport at plugin
method "logs.collect()"

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1767943/+subscriptions