New mklog script

Message ID 92be53d4-51ab-ee7f-53a5-68bdb6ed9a2c@suse.cz
State New
Headers show
Series
  • New mklog script
Related show

Commit Message

Martin Liška May 15, 2020, 8:59 a.m.
Hi.

Since we moved to git world and we're in the preparation for ChangeLog messages
being in git commit messages, I think it's the right time to also simplify mklog
script.

I'm sending a new version (which should eventually replace contrib/mklog and contrib/mklog.pl).
Changes made in the version:

- the script uses unifdiff - it rapidly simplifies parsing of the '+-!' lines that is done
   in contrib/mklog
- no author nor date stamp is used - that all can be get from git
- --inline option is not supported - I don't see a use-case for it now
- the new script has a unit tests (just few of them for now)

I compares results in between the old Python script for last 80 commits and it's very close,
in some cases it does even better.

I'm planning to maintain and improve the script for the future.

Thoughts?
Martin

Comments

Thomas Rodgers via Gcc-patches May 15, 2020, 10:58 a.m. | #1
On Fri, 2020-05-15 at 10:59 +0200, Martin Liška wrote:
> Hi.

> 

> Since we moved to git world and we're in the preparation for

> ChangeLog messages

> being in git commit messages, I think it's the right time to also

> simplify mklog

> script.

> 

> I'm sending a new version (which should eventually replace

> contrib/mklog and contrib/mklog.pl).

> Changes made in the version:

> 

> - the script uses unifdiff - it rapidly simplifies parsing of the '+-

> !' lines that is done

>    in contrib/mklog

> - no author nor date stamp is used - that all can be get from git

> - --inline option is not supported - I don't see a use-case for it

> now

> - the new script has a unit tests (just few of them for now)

> 

> I compares results in between the old Python script for last 80

> commits and it's very close,

> in some cases it does even better.

> 

> I'm planning to maintain and improve the script for the future.

> 

> Thoughts?

> Martin


> +class TestMklog(unittest.TestCase):

> +    def test_macro_definition(self):

> +        changelog = generate_changelog(PATCH1)

> +        assert changelog == EXPECTED1

> +

> +    def test_changed_argument(self):

> +        changelog = generate_changelog(PATCH2)

> +        assert changelog == EXPECTED2

> +

> +    def test_enum_and_struct(self):

> +        changelog = generate_changelog(PATCH3)

> +        assert changelog == EXPECTED3

> +

> +    def test_no_function(self):

> +        changelog = generate_changelog(PATCH3, True)

> +        assert changelog == EXPECTED3B


Use self.assertEqual(a, b) rather than assert a == b, so that if it
fails you get a multiline diff:

e.g.:

import unittest

class TestMklog(unittest.TestCase):
    def test_macro_definition(self):
        self.assertEqual('''
first
second
third''', '''
first
SECOND
third''')

unittest.main()


has this output:

F
======================================================================
FAIL: test_macro_definition (__main__.TestMklog)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/foo.py", line 11, in test_macro_definition
    third''')
AssertionError: '\nfirst\nsecond\nthird' != '\nfirst\nSECOND\nthird'
  
  first
- second
+ SECOND
  third

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (failures=1)

which is much easier to debug than the output from assert a == b, which
is just:

F
======================================================================
FAIL: test_macro_definition (__main__.TestMklog)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/foo.py", line 11, in test_macro_definition
    third''')
AssertionError

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (failures=1)
Martin Liška May 15, 2020, 11:20 a.m. | #2
On 5/15/20 12:58 PM, David Malcolm wrote:
> On Fri, 2020-05-15 at 10:59 +0200, Martin Liška wrote:

>> Hi.

>>

>> Since we moved to git world and we're in the preparation for

>> ChangeLog messages

>> being in git commit messages, I think it's the right time to also

>> simplify mklog

>> script.

>>

>> I'm sending a new version (which should eventually replace

>> contrib/mklog and contrib/mklog.pl).

>> Changes made in the version:

>>

>> - the script uses unifdiff - it rapidly simplifies parsing of the '+-

>> !' lines that is done

>>     in contrib/mklog

>> - no author nor date stamp is used - that all can be get from git

>> - --inline option is not supported - I don't see a use-case for it

>> now

>> - the new script has a unit tests (just few of them for now)

>>

>> I compares results in between the old Python script for last 80

>> commits and it's very close,

>> in some cases it does even better.

>>

>> I'm planning to maintain and improve the script for the future.

>>

>> Thoughts?

>> Martin

> 

>> +class TestMklog(unittest.TestCase):

>> +    def test_macro_definition(self):

>> +        changelog = generate_changelog(PATCH1)

>> +        assert changelog == EXPECTED1

>> +

>> +    def test_changed_argument(self):

>> +        changelog = generate_changelog(PATCH2)

>> +        assert changelog == EXPECTED2

>> +

>> +    def test_enum_and_struct(self):

>> +        changelog = generate_changelog(PATCH3)

>> +        assert changelog == EXPECTED3

>> +

>> +    def test_no_function(self):

>> +        changelog = generate_changelog(PATCH3, True)

>> +        assert changelog == EXPECTED3B


Thank you David for review.

However I see the same output for both operator== and assertEqual. Probably
because of usage of pytest version 4?

assertEqual:

$ pytest contrib/test_mklog_ng.py
Test session starts (platform: linux, Python 3.8.2, pytest 4.6.9, pytest-sugar 0.9.3)
benchmark: 3.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/marxin/Programming/gcc
plugins: xdist-1.32.0, sugar-0.9.3, forked-1.1.3, benchmark-3.2.3, aspectlib-1.5.0, cov-2.8.1, flake8-1.0.5
collecting ...
  contrib/test_mklog_ng.py ✓                                                                                                                                                                                                                                       25% ██▌

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestMklog.test_enum_and_struct ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_mklog_ng.TestMklog testMethod=test_enum_and_struct>

     def test_enum_and_struct(self):
         changelog = generate_changelog(PATCH3)
>       self.assertEqual(changelog, EXPECTED3)

E       AssertionError: 'libc[23 chars]clude/cpplib.h (enum c_lang):\n\t(struct cpp_options):\n\n' != 'libc[23 chars]clude/cppli22b.h (enum c_lang):\n\t(struct cpp_optio44ns):\n\n'
E         libcpp/ChangeLog:
E
E       - 	* include/cpplib.h (enum c_lang):
E       + 	* include/cppli22b.h (enum c_lang):
E       ? 	               ++
E       - 	(struct cpp_options):
E       + 	(struct cpp_optio44ns):
E       ? 	                 ++

contrib/test_mklog_ng.py:154: AssertionError

operator==:

pytest contrib/test_mklog_ng.py
Test session starts (platform: linux, Python 3.8.2, pytest 4.6.9, pytest-sugar 0.9.3)
benchmark: 3.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/marxin/Programming/gcc
plugins: xdist-1.32.0, sugar-0.9.3, forked-1.1.3, benchmark-3.2.3, aspectlib-1.5.0, cov-2.8.1, flake8-1.0.5
collecting ...
  contrib/test_mklog_ng.py ✓                                                                                                                                                                                                                                       25% ██▌

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestMklog.test_enum_and_struct ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_mklog_ng.TestMklog testMethod=test_enum_and_struct>

     def test_enum_and_struct(self):
         changelog = generate_changelog(PATCH3)
>       assert changelog == EXPECTED3

E       AssertionError: assert 'libcpp/Chang...options):\n\n' == 'libcpp/Change...tio44ns):\n\n'
E           libcpp/ChangeLog:
E
E         - 	* include/cpplib.h (enum c_lang):
E         + 	* include/cppli22b.h (enum c_lang):
E         ? 	               ++
E         - 	(struct cpp_options):
E         + 	(struct cpp_optio44ns):...
E
E         ...Full output truncated (3 lines hidden), use '-vv' to show

Martin

> 

> Use self.assertEqual(a, b) rather than assert a == b, so that if it

> fails you get a multiline diff:

> 

> e.g.:

> 

> import unittest

> 

> class TestMklog(unittest.TestCase):

>      def test_macro_definition(self):

>          self.assertEqual('''

> first

> second

> third''', '''

> first

> SECOND

> third''')

> 

> unittest.main()

> 

> 

> has this output:

> 

> F

> ======================================================================

> FAIL: test_macro_definition (__main__.TestMklog)

> ----------------------------------------------------------------------

> Traceback (most recent call last):

>    File "/tmp/foo.py", line 11, in test_macro_definition

>      third''')

> AssertionError: '\nfirst\nsecond\nthird' != '\nfirst\nSECOND\nthird'

>    

>    first

> - second

> + SECOND

>    third

> 

> ----------------------------------------------------------------------

> Ran 1 test in 0.000s

> 

> FAILED (failures=1)

> 

> which is much easier to debug than the output from assert a == b, which

> is just:

> 

> F

> ======================================================================

> FAIL: test_macro_definition (__main__.TestMklog)

> ----------------------------------------------------------------------

> Traceback (most recent call last):

>    File "/tmp/foo.py", line 11, in test_macro_definition

>      third''')

> AssertionError

> 

> ----------------------------------------------------------------------

> Ran 1 test in 0.000s

> 

> FAILED (failures=1)

>
Thomas Rodgers via Gcc-patches May 15, 2020, 12:42 p.m. | #3
On Fri, May 15, 2020 at 10:59:56AM +0200, Martin Liška wrote:
> Hi.

> 

> Since we moved to git world and we're in the preparation for ChangeLog messages

> being in git commit messages, I think it's the right time to also simplify mklog

> script.

> 

> I'm sending a new version (which should eventually replace contrib/mklog and contrib/mklog.pl).

> Changes made in the version:

> 

> - the script uses unifdiff - it rapidly simplifies parsing of the '+-!' lines that is done

>   in contrib/mklog


Nice!

> - no author nor date stamp is used - that all can be get from git


This is good.

> - --inline option is not supported - I don't see a use-case for it now


I actually use mklog -i all the time.  But I can work around it if it
disappears.

--
Marek Polacek • Red Hat, Inc. • 300 A St, Boston, MA
Thomas Rodgers via Gcc-patches May 15, 2020, 1:11 p.m. | #4
On Fri, 2020-05-15 at 13:20 +0200, Martin Liška wrote:
> On 5/15/20 12:58 PM, David Malcolm wrote:

> > On Fri, 2020-05-15 at 10:59 +0200, Martin Liška wrote:

> > > Hi.

> > > 

> > > Since we moved to git world and we're in the preparation for

> > > ChangeLog messages

> > > being in git commit messages, I think it's the right time to also

> > > simplify mklog

> > > script.

> > > 

> > > I'm sending a new version (which should eventually replace

> > > contrib/mklog and contrib/mklog.pl).

> > > Changes made in the version:

> > > 

> > > - the script uses unifdiff - it rapidly simplifies parsing of the

> > > '+-

> > > !' lines that is done

> > >     in contrib/mklog

> > > - no author nor date stamp is used - that all can be get from git

> > > - --inline option is not supported - I don't see a use-case for

> > > it

> > > now

> > > - the new script has a unit tests (just few of them for now)

> > > 

> > > I compares results in between the old Python script for last 80

> > > commits and it's very close,

> > > in some cases it does even better.

> > > 

> > > I'm planning to maintain and improve the script for the future.

> > > 

> > > Thoughts?

> > > Martin

> > > +class TestMklog(unittest.TestCase):

> > > +    def test_macro_definition(self):

> > > +        changelog = generate_changelog(PATCH1)

> > > +        assert changelog == EXPECTED1

> > > +

> > > +    def test_changed_argument(self):

> > > +        changelog = generate_changelog(PATCH2)

> > > +        assert changelog == EXPECTED2

> > > +

> > > +    def test_enum_and_struct(self):

> > > +        changelog = generate_changelog(PATCH3)

> > > +        assert changelog == EXPECTED3

> > > +

> > > +    def test_no_function(self):

> > > +        changelog = generate_changelog(PATCH3, True)

> > > +        assert changelog == EXPECTED3B

> 

> Thank you David for review.

> 

> However I see the same output for both operator== and assertEqual.

> Probably

> because of usage of pytest version 4?


Ah, yes.  pytest does "magical" things with frame inspection IIRC to
scrape the locals out of the failing python stack frame.

Dave
Martin Liška May 15, 2020, 1:12 p.m. | #5
On 5/15/20 2:42 PM, Marek Polacek wrote:
> I actually use mklog -i all the time.  But I can work around it if it

> disappears.


Ah, I can see a consumer.
There's an updated version that supports that.

For the future, will you still use the option? Wouldn't be better
to put the ChangeLog content directly to commit message? Note
that you won't have to copy the entries to a particular ChangeLog file.

Martin
From d7d5e3aa7450449a8b0cb30d6bf485538990ea3f Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>

Date: Fri, 15 May 2020 00:44:07 +0200
Subject: [PATCH] Add mklog-ng.py and gcc-mklog git alias.

contrib/ChangeLog:

	* gcc-git-customization.sh: Add gcc-mklog alias.
	* mklog_ng.py: New file.
	* test_mklog_ng.py: New file.
---
 contrib/gcc-git-customization.sh |   2 +
 contrib/mklog_ng.py              | 209 +++++++++++++++++++++++++++++++
 contrib/test_mklog_ng.py         | 158 +++++++++++++++++++++++
 3 files changed, 369 insertions(+)
 create mode 100755 contrib/mklog_ng.py
 create mode 100755 contrib/test_mklog_ng.py

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index a932bf8c06a..b7b97327be3 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -25,6 +25,8 @@ git config alias.svn-rev '!f() { rev=$1; shift; git log --all --grep="^From-SVN:
 git config alias.gcc-descr \!"f() { if test \${1:-no} = --full; then c=\${2:-master}; r=\$(git describe --all --abbrev=40 --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-,r,p'); expr match \${r:-no} '^r[0-9]\\+\$' >/dev/null && r=\${r}-0-g\$(git rev-parse \${2:-master}); else c=\${1:-master}; r=\$(git describe --all --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)-\\([0-9]\\+\\)-g[0-9a-f]*\$,r\\2-\\3,p;s,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)\$,r\\2-0,p'); fi; if test -n \$r; then o=\$(git config --get gcc-config.upstream); rr=\$(echo \$r | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\\(-g[0-9a-f]\\+\\)\\?\$,\\1,p'); if git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$rr >/dev/null; then m=releases/gcc-\$rr; else m=master; fi; git merge-base --is-ancestor \$c \${o:-origin}/\$m && \echo \${r}; fi; }; f"
 git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream); r=\$(echo \$1 | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\$,\\1,p'); n=\$(echo \$1 | sed -n 's,^r[0-9]\\+-\\([0-9]\\+\\)\$,\\1,p'); test -z \$r && echo Invalid id \$1 && exit 1; h=\$(git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$r); test -z \$h && h=\$(git rev-parse --verify --quiet \${o:-origin}/master); p=\$(git describe --all --match 'basepoints/gcc-'\$r \$h | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+-\\([0-9]\\+\\)-g[0-9a-f]*\$,\\2,p;s,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+\$,0,p'); git rev-parse --verify \$h~\$(expr \$p - \$n); }; f"
 
+git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog_ng.py" $@; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
diff --git a/contrib/mklog_ng.py b/contrib/mklog_ng.py
new file mode 100755
index 00000000000..8dca6dbeef0
--- /dev/null
+++ b/contrib/mklog_ng.py
@@ -0,0 +1,209 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import argparse
+import os
+import re
+import sys
+import tempfile
+
+from unidiff import PatchSet
+
+pr_regex = re.compile(r'(\/(\/|\*)|[Cc*!])\s+(?P<pr>PR [a-z+-]+\/[0-9]+)')
+identifier_regex = re.compile(r'^([a-zA-Z0-9_#].*)')
+comment_regex = re.compile(r'^\/\*')
+struct_regex = re.compile(r'^((class|struct|union|enum)\s+[a-zA-Z0-9_]+)')
+macro_regex = re.compile(r'#\s*(define|undef)\s+([a-zA-Z0-9_]+)')
+super_macro_regex = re.compile(r'^DEF[A-Z0-9_]+\s*\(([a-zA-Z0-9_]+)')
+fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
+template_and_param_regex = re.compile(r'<[^<>]*>')
+
+function_extensions = set(['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def'])
+
+help_message = """\
+Generate ChangeLog template for PATCH.
+PATCH must be generated using diff(1)'s -up or -cp options
+(or their equivalent in git).
+"""
+
+inline_message = """\
+Prepends ChangeLog to PATCH.
+If PATCH is not stdin, modifies PATCH in-place,
+otherwise writes to stdout.'
+"""
+
+script_folder = os.path.realpath(__file__)
+gcc_root = os.path.dirname(os.path.dirname(script_folder))
+
+
+def find_changelog(path):
+    folder = os.path.split(path)[0]
+    while True:
+        if os.path.exists(os.path.join(gcc_root, folder, 'ChangeLog')):
+            return folder
+        folder = os.path.dirname(folder)
+        if folder == '':
+            return folder
+    raise AssertionError()
+
+
+def extract_function_name(line):
+    if comment_regex.match(line):
+        return None
+    m = struct_regex.search(line)
+    if m:
+        # Struct declaration
+        return m.group(1)
+    m = macro_regex.search(line)
+    if m:
+        # Macro definition
+        return m.group(2)
+    m = super_macro_regex.search(line)
+    if m:
+        # Supermacro
+        return m.group(1)
+    m = fn_regex.search(line)
+    if m:
+        # Discard template and function parameters.
+        fn = m.group(1)
+        fn = re.sub(template_and_param_regex, '', fn)
+        return fn.rstrip()
+    return None
+
+
+def try_add_function(functions, line):
+    fn = extract_function_name(line)
+    if fn and fn not in functions:
+        functions.append(fn)
+    return bool(fn)
+
+
+def generate_changelog(data, no_functions=False):
+    changelogs = {}
+    sorted_changelogs = []
+    prs = []
+    out = ''
+    diff = PatchSet(data)
+
+    for file in diff:
+        changelog = find_changelog(file.path)
+        if changelog not in changelogs:
+            changelogs[changelog] = []
+            sorted_changelogs.append(changelog)
+        changelogs[changelog].append(file)
+
+        if 'testsuite' in file.path and file.is_added_file:
+            for line in list(file)[0]:
+                m = pr_regex.search(line.value)
+                if m:
+                    pr = m.group('pr')
+                    if pr not in prs:
+                        prs.append(pr)
+                else:
+                    break
+
+    for changelog in sorted_changelogs:
+        files = changelogs[changelog]
+        out += '%s:\n' % os.path.join(changelog, 'ChangeLog')
+        out += '\n'
+        for pr in prs:
+            out += '\t%s\n' % pr
+        for file in files:
+            assert file.path.startswith(changelog)
+            in_tests = 'testsuite' in changelog or 'testsuite' in file.path
+            relative_path = file.path[len(changelog):].lstrip('/')
+            functions = []
+            if file.is_added_file:
+                msg = 'New test' if in_tests else 'New file'
+                out += '\t* %s: %s.\n' % (relative_path, msg)
+            elif file.is_removed_file:
+                out += '\t* %s: Removed.\n' % (relative_path)
+            else:
+                if not no_functions:
+                    for hunk in file:
+                        # Do not add function names for testsuite files
+                        extension = os.path.splitext(relative_path)[1]
+                        if not in_tests and extension in function_extensions:
+                            last_fn = None
+                            modified_visited = False
+                            success = False
+                            for line in hunk:
+                                m = identifier_regex.match(line.value)
+                                if line.is_added or line.is_removed:
+                                    if not line.value.strip():
+                                        continue
+                                    modified_visited = True
+                                    if m and try_add_function(functions,
+                                                              m.group(1)):
+                                        last_fn = None
+                                        success = True
+                                elif line.is_context:
+                                    if last_fn and modified_visited:
+                                        try_add_function(functions, last_fn)
+                                        last_fn = None
+                                        modified_visited = False
+                                        success = True
+                                    elif m:
+                                        last_fn = m.group(1)
+                                        modified_visited = False
+                            if not success:
+                                try_add_function(functions,
+                                                 hunk.section_header)
+                if functions:
+                    out += '\t* %s (%s):\n' % (relative_path, functions[0])
+                    for fn in functions[1:]:
+                        out += '\t(%s):\n' % fn
+                else:
+                    out += '\t* %s:\n' % relative_path
+        out += '\n'
+    return out
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=help_message)
+    parser.add_argument('input', nargs='?',
+                        help='Patch file (or missing, read standard input)')
+    parser.add_argument('-s', '--no-functions', action='store_true',
+                        help='Do not generate function names in ChangeLogs')
+    parser.add_argument('-i', '--inline', action='store_true',
+                        help=inline_message)
+    args = parser.parse_args()
+    if args.input == '-':
+        args.input = None
+
+    input = open(args.input) if args.input else sys.stdin
+    data = input.read()
+    output = generate_changelog(data, args.no_functions)
+
+    if args.inline and args.input:
+        with tempfile.NamedTemporaryFile(mode='w', delete=False) as f:
+            f.write(output)
+            f.write(data)
+        os.rename(f.name, args.input)
+    else:
+        print(output, end='')
diff --git a/contrib/test_mklog_ng.py b/contrib/test_mklog_ng.py
new file mode 100755
index 00000000000..6e68277b1bb
--- /dev/null
+++ b/contrib/test_mklog_ng.py
@@ -0,0 +1,158 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import unittest
+
+from mklog_ng import generate_changelog
+
+PATCH1 = '''\
+diff --git a/gcc/config/riscv/riscv.h b/gcc/config/riscv/riscv.h
+index 567c23380fe..e6209ede9d6 100644
+--- a/gcc/config/riscv/riscv.h
++++ b/gcc/config/riscv/riscv.h
+@@ -920,6 +920,7 @@ extern unsigned riscv_stack_boundary;
+ #define SHIFT_RS1 15
+ #define SHIFT_IMM 20
+ #define IMM_BITS 12
++#define C_S_BITS 5
+ #define C_SxSP_BITS 6
+ 
+ #define IMM_REACH (1LL << IMM_BITS)
+@@ -929,6 +930,10 @@ extern unsigned riscv_stack_boundary;
+ #define SWSP_REACH (4LL << C_SxSP_BITS)
+ #define SDSP_REACH (8LL << C_SxSP_BITS)
+ 
++/* This is the maximum value that can be represented in a compressed load/store
++   offset (an unsigned 5-bit value scaled by 4).  */
++#define CSW_MAX_OFFSET ((4LL << C_S_BITS) - 1) & ~3
++
+ /* Called from RISCV_REORG, this is defined in riscv-sr.c.  */
+ 
+ extern void riscv_remove_unneeded_save_restore_calls (void);
+
+'''
+
+EXPECTED1 = '''\
+gcc/ChangeLog:
+
+	* config/riscv/riscv.h (C_S_BITS):
+	(CSW_MAX_OFFSET):
+
+'''
+
+PATCH2 = '''\
+diff --git a/gcc/targhooks.h b/gcc/targhooks.h
+index 9704d23f1db..b572a36e8cf 100644
+--- a/gcc/targhooks.h
++++ b/gcc/targhooks.h
+@@ -120,7 +120,7 @@ extern bool default_empty_mask_is_expensive (unsigned);
+ extern void *default_init_cost (class loop *);
+ extern unsigned default_add_stmt_cost (class vec_info *, void *, int,
+ 				       enum vect_cost_for_stmt,
+-				       class _stmt_vec_info *, int,
++				       class _stmt_vec_info *, tree, int,
+ 				       enum vect_cost_model_location);
+ extern void default_finish_cost (void *, unsigned *, unsigned *, unsigned *);
+ extern void default_destroy_cost_data (void *);
+@@ -186,6 +186,7 @@ extern tree default_emutls_var_init (tree, tree, tree);
+ extern unsigned int default_hard_regno_nregs (unsigned int, machine_mode);
+ extern bool default_hard_regno_scratch_ok (unsigned int);
+ extern bool default_mode_dependent_address_p (const_rtx, addr_space_t);
++extern bool default_new_address_profitable_p (rtx, rtx_insn *, rtx);
+ extern bool default_target_option_valid_attribute_p (tree, tree, tree, int);
+ extern bool default_target_option_pragma_parse (tree, tree);
+ extern bool default_target_can_inline_p (tree, tree);
+
+'''
+
+EXPECTED2 = '''\
+gcc/ChangeLog:
+
+	* targhooks.h (default_add_stmt_cost):
+	(default_new_address_profitable_p):
+
+'''
+
+PATCH3 = '''\
+diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
+index 2b1e33f94ae..7f47402f9b9 100644
+--- a/libcpp/include/cpplib.h
++++ b/libcpp/include/cpplib.h
+@@ -173,7 +173,7 @@ enum c_lang {CLK_GNUC89 = 0, CLK_GNUC99, CLK_GNUC11, CLK_GNUC17, CLK_GNUC2X,
+ 	     CLK_STDC2X,
+ 	     CLK_GNUCXX, CLK_CXX98, CLK_GNUCXX11, CLK_CXX11,
+ 	     CLK_GNUCXX14, CLK_CXX14, CLK_GNUCXX17, CLK_CXX17,
+-	     CLK_GNUCXX2A, CLK_CXX2A, CLK_ASM};
++	     CLK_GNUCXX20, CLK_CXX20, CLK_ASM};
+ 
+ /* Payload of a NUMBER, STRING, CHAR or COMMENT token.  */
+ struct GTY(()) cpp_string {
+@@ -484,7 +484,7 @@ struct cpp_options
+   /* Nonzero for C2X decimal floating-point constants.  */
+   unsigned char dfp_constants;
+ 
+-  /* Nonzero for C++2a __VA_OPT__ feature.  */
++  /* Nonzero for C++20 __VA_OPT__ feature.  */
+   unsigned char va_opt;
+ 
+   /* Nonzero for the '::' token.  */
+
+'''
+
+EXPECTED3 = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h (enum c_lang):
+	(struct cpp_options):
+
+'''
+
+EXPECTED3B = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h:
+
+'''
+
+
+class TestMklog(unittest.TestCase):
+    def test_macro_definition(self):
+        changelog = generate_changelog(PATCH1)
+        assert changelog == EXPECTED1
+
+    def test_changed_argument(self):
+        changelog = generate_changelog(PATCH2)
+        assert changelog == EXPECTED2
+
+    def test_enum_and_struct(self):
+        changelog = generate_changelog(PATCH3)
+        assert changelog == EXPECTED3
+
+    def test_no_function(self):
+        changelog = generate_changelog(PATCH3, True)
+        assert changelog == EXPECTED3B
-- 
2.26.2
Thomas Rodgers via Gcc-patches May 15, 2020, 1:22 p.m. | #6
On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:
> On 5/15/20 2:42 PM, Marek Polacek wrote:

> > I actually use mklog -i all the time.  But I can work around it if it

> > disappears.

> 

> Ah, I can see a consumer.

> There's an updated version that supports that.

> 

> For the future, will you still use the option? Wouldn't be better

> to put the ChangeLog content directly to commit message? Note

> that you won't have to copy the entries to a particular ChangeLog file.


The way I do it is to generate a patch using format-patch, use mklog -i
on it, then add the ChangeLog entry to the commit message via commit --amend.

Anything that has to do with ChangeLogs is pointless make-work, so the less
I have to do, the better.  ;-)

Marek
Thomas Rodgers via Gcc-patches May 15, 2020, 3:06 p.m. | #7
On 5/15/20 2:59 AM, Martin Liška wrote:
> Hi.

> 

> Since we moved to git world and we're in the preparation for ChangeLog 

> messages

> being in git commit messages, I think it's the right time to also 

> simplify mklog

> script.

> 

> I'm sending a new version (which should eventually replace contrib/mklog 

> and contrib/mklog.pl).

> Changes made in the version:

> 

> - the script uses unifdiff - it rapidly simplifies parsing of the '+-!' 

> lines that is done

>    in contrib/mklog

> - no author nor date stamp is used - that all can be get from git

> - --inline option is not supported - I don't see a use-case for it now

> - the new script has a unit tests (just few of them for now)

> 

> I compares results in between the old Python script for last 80 commits 

> and it's very close,

> in some cases it does even better.

> 

> I'm planning to maintain and improve the script for the future.

> 

> Thoughts?


It's pretty nice.  I have a script of my own that does the same thing
in a slightly different way.  Here's an example of its output:
https://gcc.gnu.org/pipermail/gcc-patches/attachments/20200323/5437db5a/attachment-0001.bin

I find this format more helpful for the reasons below so unless your
script can be tweaked to do something similar I'd like to be able to
continue to use mine going forward with the new infrastructure.

As for my comments on mklog_ng.py: In the one test I did the script
produced a single long ChangeLog entry with all the files in the diff
I gave it, tests and all, in alphabetical order.  The script fills in
"New test." for new tests.  The rest has to be edited as one would
expect.

I would find the output easier to work with if it a) grouped files by
"subsystem" corresponding to each ChangeLog directory (and if it also
identified each subsystem), b) put the testsuite section last, and
(as a bonus) c) grouped all new files in each section together.

First, I find this logical grouping helpful in thinking about how
the changes are structured (e.g., would it make sense to restructure
them or break things up to reduce coupling and make review easier),
and whom they need to be reviewed by.

Second, this is the grouping I'm already used to from my own script
(so YMMV here of course).

Finally, my script also looks up bugs in Bugzilla and adds a line with
each bug number and its Summary at the top of the patch.  This helps me
double-check the spelling of the bug id(s) in case I transpose digits
etc.

Martin

PS My script modifies the patch file in place: it adds the ChangeLog
section if it doesn't exist yet, but it doesn't do anything it does.
I'd love for it to check the existing ChangeLog if it exists and
update it when it finds differences between it and the latest patch
that aren't reflected there.

Without this, each time a patch changes I have to review the entry
and update it as necessary.  That makes it too easy to miss things.
Martin Liška May 15, 2020, 3:38 p.m. | #8
On 5/15/20 3:22 PM, Marek Polacek wrote:
> On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:

>> On 5/15/20 2:42 PM, Marek Polacek wrote:

>>> I actually use mklog -i all the time.  But I can work around it if it

>>> disappears.

>>

>> Ah, I can see a consumer.

>> There's an updated version that supports that.

>>

>> For the future, will you still use the option? Wouldn't be better

>> to put the ChangeLog content directly to commit message? Note

>> that you won't have to copy the entries to a particular ChangeLog file.

> 

> The way I do it is to generate a patch using format-patch, use mklog -i

> on it, then add the ChangeLog entry to the commit message via commit --amend.


Hmm, you can do much better with:

$ git diff | ./contrib/mklog > changelog && git commit -a -t changelog

Or for an already created commit you can do:

$ git diff HEAD~ | ./contrib/mklog > changelog && git commit -a --amend -e -F changelog

That said, I believe usage of -i is legacy.

Martin

> 

> Anything that has to do with ChangeLogs is pointless make-work, so the less

> I have to do, the better.  ;-)

> 

> Marek

>
Martin Liška May 19, 2020, 8:11 a.m. | #9
On 5/15/20 5:06 PM, Martin Sebor wrote:
> On 5/15/20 2:59 AM, Martin Liška wrote:

>> Hi.

>>

>> Since we moved to git world and we're in the preparation for ChangeLog messages

>> being in git commit messages, I think it's the right time to also simplify mklog

>> script.

>>

>> I'm sending a new version (which should eventually replace contrib/mklog and contrib/mklog.pl).

>> Changes made in the version:

>>

>> - the script uses unifdiff - it rapidly simplifies parsing of the '+-!' lines that is done

>>    in contrib/mklog

>> - no author nor date stamp is used - that all can be get from git

>> - --inline option is not supported - I don't see a use-case for it now

>> - the new script has a unit tests (just few of them for now)

>>

>> I compares results in between the old Python script for last 80 commits and it's very close,

>> in some cases it does even better.

>>

>> I'm planning to maintain and improve the script for the future.

>>

>> Thoughts?


Hello Martin.

I welcome the feedback. Leitmotif of the gcc-changelog changes is to simplify
scripts used by individual contributors. I must confess I have a special script
that takes content of changelog entries, applies them to individual files, or
appends Backported header to them. All that should be gone and only git messages
will be used.

> 

> It's pretty nice.  I have a script of my own that does the same thing

> in a slightly different way.  Here's an example of its output:

> https://gcc.gnu.org/pipermail/gcc-patches/attachments/20200323/5437db5a/attachment-0001.bin

> 

> I find this format more helpful for the reasons below so unless your

> script can be tweaked to do something similar I'd like to be able to

> continue to use mine going forward with the new infrastructure.


Let's extend the contrib script.

> 

> As for my comments on mklog_ng.py: In the one test I did the script

> produced a single long ChangeLog entry with all the files in the diff

> I gave it, tests and all, in alphabetical order.  The script fills in

> "New test." for new tests.  The rest has to be edited as one would

> expect.

> 

> I would find the output easier to work with if it a) grouped files by

> "subsystem" corresponding to each ChangeLog directory (and if it also


The script does that, let's consider a patch 'p':

./contrib/mklog_ng.py p
gcc/ChangeLog:

	* ipa-icf.c:
	* varasm.c:

gcc/testsuite/ChangeLog:

	* gcc.dg/pr40209.c:

The idea is to put changes in the order in which it appears in a patch.
Because normal work flow is to follow a patch and fulfill ChangeLog entries.
As seen in the patch, changes in varasm.c follow the change in
gcc/testsuite/, which can't be handled as varasm.c needs to be documented
in gcc/ChangeLog.

> identified each subsystem), b) put the testsuite section last, and

> (as a bonus)


Good idea, I'll do it!

> c) grouped all new files in each section together.


Likewise here. And what about deleted files at the very end of a section?

> 

> First, I find this logical grouping helpful in thinking about how

> the changes are structured (e.g., would it make sense to restructure

> them or break things up to reduce coupling and make review easier),

> and whom they need to be reviewed by.


Fully yes, please see the updated patch and tell me what's missing.

> 

> Second, this is the grouping I'm already used to from my own script

> (so YMMV here of course).

> 

> Finally, my script also looks up bugs in Bugzilla and adds a line with

> each bug number and its Summary at the top of the patch.  This helps me

> double-check the spelling of the bug id(s) in case I transpose digits

> etc.


Can you please share how do you do it? It would be easy to add it.

> 

> Martin

> 

> PS My script modifies the patch file in place: it adds the ChangeLog

> section if it doesn't exist yet, but it doesn't do anything it does.

> I'd love for it to check the existing ChangeLog if it exists and

> update it when it finds differences between it and the latest patch

> that aren't reflected there.


Well, what about using more git approach? I mean putting all you need
into a commit message.

> 

> Without this, each time a patch changes I have to review the entry

> and update it as necessary.  That makes it too easy to miss things.


What you can use for a git commit is 'git gcc-verify' which will
inform you about missing (undocumented) Changes.

Martin
diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c
index aab79492357..f0df1002488 100644
--- a/gcc/ipa-icf.c
+++ b/gcc/ipa-icf.c
@@ -1,5 +1,7 @@
 
 
+
+
 /* Interprocedural Identical Code Folding pass
    Copyright (C) 2014-2020 Free Software Foundation, Inc.
 
diff --git a/gcc/testsuite/gcc.dg/pr40209.c b/gcc/testsuite/gcc.dg/pr40209.c
index 4e77df5c2e6..c23d69d1f1b 100644
--- a/gcc/testsuite/gcc.dg/pr40209.c
+++ b/gcc/testsuite/gcc.dg/pr40209.c
@@ -1,6 +1,8 @@
 /* { dg-do compile } */
 /* { dg-options "-O2 -fprofile-use -fopt-info -Wno-missing-profile" } */
 
+
+
 void process(const char *s);
 
 struct BaseHolder {
diff --git a/gcc/varasm.c b/gcc/varasm.c
index f062e48071f..fd3c7ca8cf3 100644
--- a/gcc/varasm.c
+++ b/gcc/varasm.c
@@ -1,3 +1,5 @@
+
+
 /* Output variables, constants and external declarations, for GNU compiler.
    Copyright (C) 1987-2020 Free Software Foundation, Inc.
From 58719696e8b1acc06d788c0db069c4194b61e56f Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>

Date: Fri, 15 May 2020 00:44:07 +0200
Subject: [PATCH] New mklog script.

contrib/ChangeLog:

2020-05-15  Martin Liska  <mliska@suse.cz>

	* gcc-git-customization.sh: Add
	alias.gcc-mklog new hook.
	* mklog_ng.py: New file.
	* test_mklog_ng.py: New file.
---
 contrib/gcc-git-customization.sh |   2 +
 contrib/mklog_ng.py              | 200 ++++++++++++++++++++++
 contrib/test_mklog_ng.py         | 275 +++++++++++++++++++++++++++++++
 gcc/ipa-icf.c                    |   2 +
 4 files changed, 479 insertions(+)
 create mode 100755 contrib/mklog_ng.py
 create mode 100755 contrib/test_mklog_ng.py

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index a932bf8c06a..b7b97327be3 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -25,6 +25,8 @@ git config alias.svn-rev '!f() { rev=$1; shift; git log --all --grep="^From-SVN:
 git config alias.gcc-descr \!"f() { if test \${1:-no} = --full; then c=\${2:-master}; r=\$(git describe --all --abbrev=40 --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-,r,p'); expr match \${r:-no} '^r[0-9]\\+\$' >/dev/null && r=\${r}-0-g\$(git rev-parse \${2:-master}); else c=\${1:-master}; r=\$(git describe --all --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)-\\([0-9]\\+\\)-g[0-9a-f]*\$,r\\2-\\3,p;s,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)\$,r\\2-0,p'); fi; if test -n \$r; then o=\$(git config --get gcc-config.upstream); rr=\$(echo \$r | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\\(-g[0-9a-f]\\+\\)\\?\$,\\1,p'); if git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$rr >/dev/null; then m=releases/gcc-\$rr; else m=master; fi; git merge-base --is-ancestor \$c \${o:-origin}/\$m && \echo \${r}; fi; }; f"
 git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream); r=\$(echo \$1 | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\$,\\1,p'); n=\$(echo \$1 | sed -n 's,^r[0-9]\\+-\\([0-9]\\+\\)\$,\\1,p'); test -z \$r && echo Invalid id \$1 && exit 1; h=\$(git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$r); test -z \$h && h=\$(git rev-parse --verify --quiet \${o:-origin}/master); p=\$(git describe --all --match 'basepoints/gcc-'\$r \$h | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+-\\([0-9]\\+\\)-g[0-9a-f]*\$,\\2,p;s,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+\$,0,p'); git rev-parse --verify \$h~\$(expr \$p - \$n); }; f"
 
+git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog_ng.py" $@; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
diff --git a/contrib/mklog_ng.py b/contrib/mklog_ng.py
new file mode 100755
index 00000000000..228f87f0156
--- /dev/null
+++ b/contrib/mklog_ng.py
@@ -0,0 +1,200 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import argparse
+import os
+import re
+import sys
+
+from unidiff import PatchSet
+
+pr_regex = re.compile(r'(\/(\/|\*)|[Cc*!])\s+(?P<pr>PR [a-z+-]+\/[0-9]+)')
+identifier_regex = re.compile(r'^([a-zA-Z0-9_#].*)')
+comment_regex = re.compile(r'^\/\*')
+struct_regex = re.compile(r'^((class|struct|union|enum)\s+[a-zA-Z0-9_]+)')
+macro_regex = re.compile(r'#\s*(define|undef)\s+([a-zA-Z0-9_]+)')
+super_macro_regex = re.compile(r'^DEF[A-Z0-9_]+\s*\(([a-zA-Z0-9_]+)')
+fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
+template_and_param_regex = re.compile(r'<[^<>]*>')
+
+function_extensions = set(['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def'])
+
+help_message = """\
+Generate ChangeLog template for PATCH.
+PATCH must be generated using diff(1)'s -up or -cp options
+(or their equivalent in git).
+"""
+
+script_folder = os.path.realpath(__file__)
+gcc_root = os.path.dirname(os.path.dirname(script_folder))
+
+
+def find_changelog(path):
+    folder = os.path.split(path)[0]
+    while True:
+        if os.path.exists(os.path.join(gcc_root, folder, 'ChangeLog')):
+            return folder
+        folder = os.path.dirname(folder)
+        if folder == '':
+            return folder
+    raise AssertionError()
+
+
+def extract_function_name(line):
+    if comment_regex.match(line):
+        return None
+    m = struct_regex.search(line)
+    if m:
+        # Struct declaration
+        return m.group(1)
+    m = macro_regex.search(line)
+    if m:
+        # Macro definition
+        return m.group(2)
+    m = super_macro_regex.search(line)
+    if m:
+        # Supermacro
+        return m.group(1)
+    m = fn_regex.search(line)
+    if m:
+        # Discard template and function parameters.
+        fn = m.group(1)
+        fn = re.sub(template_and_param_regex, '', fn)
+        return fn.rstrip()
+    return None
+
+
+def try_add_function(functions, line):
+    fn = extract_function_name(line)
+    if fn and fn not in functions:
+        functions.append(fn)
+    return bool(fn)
+
+
+def sort_changelog_files(changed_file):
+    return (changed_file.is_added_file, changed_file.is_removed_file)
+
+
+def generate_changelog(data, no_functions=False):
+    changelogs = {}
+    changelog_list = []
+    prs = []
+    out = ''
+    diff = PatchSet(data)
+
+    for file in diff:
+        changelog = find_changelog(file.path)
+        if changelog not in changelogs:
+            changelogs[changelog] = []
+            changelog_list.append(changelog)
+        changelogs[changelog].append(file)
+
+        # Extract PR entries from newly added tests
+        if 'testsuite' in file.path and file.is_added_file:
+            for line in list(file)[0]:
+                m = pr_regex.search(line.value)
+                if m:
+                    pr = m.group('pr')
+                    if pr not in prs:
+                        prs.append(pr)
+                else:
+                    break
+
+    # sort ChangeLog so that 'testsuite' is at the end
+    for changelog in sorted(changelog_list, key=lambda x: 'testsuite' in x):
+        files = changelogs[changelog]
+        out += '%s:\n' % os.path.join(changelog, 'ChangeLog')
+        out += '\n'
+        for pr in prs:
+            out += '\t%s\n' % pr
+        # new and deleted files should be at the end
+        for file in sorted(files, key=sort_changelog_files):
+            assert file.path.startswith(changelog)
+            in_tests = 'testsuite' in changelog or 'testsuite' in file.path
+            relative_path = file.path[len(changelog):].lstrip('/')
+            functions = []
+            if file.is_added_file:
+                msg = 'New test' if in_tests else 'New file'
+                out += '\t* %s: %s.\n' % (relative_path, msg)
+            elif file.is_removed_file:
+                out += '\t* %s: Removed.\n' % (relative_path)
+            else:
+                if not no_functions:
+                    for hunk in file:
+                        # Do not add function names for testsuite files
+                        extension = os.path.splitext(relative_path)[1]
+                        if not in_tests and extension in function_extensions:
+                            last_fn = None
+                            modified_visited = False
+                            success = False
+                            for line in hunk:
+                                m = identifier_regex.match(line.value)
+                                if line.is_added or line.is_removed:
+                                    if not line.value.strip():
+                                        continue
+                                    modified_visited = True
+                                    if m and try_add_function(functions,
+                                                              m.group(1)):
+                                        last_fn = None
+                                        success = True
+                                elif line.is_context:
+                                    if last_fn and modified_visited:
+                                        try_add_function(functions, last_fn)
+                                        last_fn = None
+                                        modified_visited = False
+                                        success = True
+                                    elif m:
+                                        last_fn = m.group(1)
+                                        modified_visited = False
+                            if not success:
+                                try_add_function(functions,
+                                                 hunk.section_header)
+                if functions:
+                    out += '\t* %s (%s):\n' % (relative_path, functions[0])
+                    for fn in functions[1:]:
+                        out += '\t(%s):\n' % fn
+                else:
+                    out += '\t* %s:\n' % relative_path
+        out += '\n'
+    return out
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=help_message)
+    parser.add_argument('input', nargs='?',
+                        help='Patch file (or missing, read standard input)')
+    parser.add_argument('-s', '--no-functions', action='store_true',
+                        help='Do not generate function names in ChangeLogs')
+    args = parser.parse_args()
+    if args.input == '-':
+        args.input = None
+
+    input = open(args.input) if args.input else sys.stdin
+    data = input.read()
+    output = generate_changelog(data, args.no_functions)
+    print(output, end='')
diff --git a/contrib/test_mklog_ng.py b/contrib/test_mklog_ng.py
new file mode 100755
index 00000000000..6f626eb49e1
--- /dev/null
+++ b/contrib/test_mklog_ng.py
@@ -0,0 +1,275 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import unittest
+
+from mklog_ng import generate_changelog
+
+PATCH1 = '''\
+diff --git a/gcc/config/riscv/riscv.h b/gcc/config/riscv/riscv.h
+index 567c23380fe..e6209ede9d6 100644
+--- a/gcc/config/riscv/riscv.h
++++ b/gcc/config/riscv/riscv.h
+@@ -920,6 +920,7 @@ extern unsigned riscv_stack_boundary;
+ #define SHIFT_RS1 15
+ #define SHIFT_IMM 20
+ #define IMM_BITS 12
++#define C_S_BITS 5
+ #define C_SxSP_BITS 6
+ 
+ #define IMM_REACH (1LL << IMM_BITS)
+@@ -929,6 +930,10 @@ extern unsigned riscv_stack_boundary;
+ #define SWSP_REACH (4LL << C_SxSP_BITS)
+ #define SDSP_REACH (8LL << C_SxSP_BITS)
+ 
++/* This is the maximum value that can be represented in a compressed load/store
++   offset (an unsigned 5-bit value scaled by 4).  */
++#define CSW_MAX_OFFSET ((4LL << C_S_BITS) - 1) & ~3
++
+ /* Called from RISCV_REORG, this is defined in riscv-sr.c.  */
+ 
+ extern void riscv_remove_unneeded_save_restore_calls (void);
+
+'''
+
+EXPECTED1 = '''\
+gcc/ChangeLog:
+
+	* config/riscv/riscv.h (C_S_BITS):
+	(CSW_MAX_OFFSET):
+
+'''
+
+PATCH2 = '''\
+diff --git a/gcc/targhooks.h b/gcc/targhooks.h
+index 9704d23f1db..b572a36e8cf 100644
+--- a/gcc/targhooks.h
++++ b/gcc/targhooks.h
+@@ -120,7 +120,7 @@ extern bool default_empty_mask_is_expensive (unsigned);
+ extern void *default_init_cost (class loop *);
+ extern unsigned default_add_stmt_cost (class vec_info *, void *, int,
+ 				       enum vect_cost_for_stmt,
+-				       class _stmt_vec_info *, int,
++				       class _stmt_vec_info *, tree, int,
+ 				       enum vect_cost_model_location);
+ extern void default_finish_cost (void *, unsigned *, unsigned *, unsigned *);
+ extern void default_destroy_cost_data (void *);
+@@ -186,6 +186,7 @@ extern tree default_emutls_var_init (tree, tree, tree);
+ extern unsigned int default_hard_regno_nregs (unsigned int, machine_mode);
+ extern bool default_hard_regno_scratch_ok (unsigned int);
+ extern bool default_mode_dependent_address_p (const_rtx, addr_space_t);
++extern bool default_new_address_profitable_p (rtx, rtx_insn *, rtx);
+ extern bool default_target_option_valid_attribute_p (tree, tree, tree, int);
+ extern bool default_target_option_pragma_parse (tree, tree);
+ extern bool default_target_can_inline_p (tree, tree);
+
+'''
+
+EXPECTED2 = '''\
+gcc/ChangeLog:
+
+	* targhooks.h (default_add_stmt_cost):
+	(default_new_address_profitable_p):
+
+'''
+
+PATCH3 = '''\
+diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
+index 2b1e33f94ae..7f47402f9b9 100644
+--- a/libcpp/include/cpplib.h
++++ b/libcpp/include/cpplib.h
+@@ -173,7 +173,7 @@ enum c_lang {CLK_GNUC89 = 0, CLK_GNUC99, CLK_GNUC11, CLK_GNUC17, CLK_GNUC2X,
+ 	     CLK_STDC2X,
+ 	     CLK_GNUCXX, CLK_CXX98, CLK_GNUCXX11, CLK_CXX11,
+ 	     CLK_GNUCXX14, CLK_CXX14, CLK_GNUCXX17, CLK_CXX17,
+-	     CLK_GNUCXX2A, CLK_CXX2A, CLK_ASM};
++	     CLK_GNUCXX20, CLK_CXX20, CLK_ASM};
+ 
+ /* Payload of a NUMBER, STRING, CHAR or COMMENT token.  */
+ struct GTY(()) cpp_string {
+@@ -484,7 +484,7 @@ struct cpp_options
+   /* Nonzero for C2X decimal floating-point constants.  */
+   unsigned char dfp_constants;
+ 
+-  /* Nonzero for C++2a __VA_OPT__ feature.  */
++  /* Nonzero for C++20 __VA_OPT__ feature.  */
+   unsigned char va_opt;
+ 
+   /* Nonzero for the '::' token.  */
+
+'''
+
+EXPECTED3 = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h (enum c_lang):
+	(struct cpp_options):
+
+'''
+
+EXPECTED3B = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h:
+
+'''
+
+PATCH4 = '''\
+diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c
+index aab79492357..f0df1002488 100644
+--- a/gcc/ipa-icf.c
++++ b/gcc/ipa-icf.c
+@@ -1,5 +1,7 @@
+ 
+ 
++
++
+ /* Interprocedural Identical Code Folding pass
+    Copyright (C) 2014-2020 Free Software Foundation, Inc.
+ 
+diff --git a/gcc/testsuite/gcc.dg/pr32374.c b/gcc/testsuite/gcc.dg/pr32374.c
+deleted file mode 100644
+index de15d559f5b..00000000000
+--- a/gcc/testsuite/gcc.dg/pr32374.c
++++ /dev/null
+@@ -1,20 +0,0 @@
+-/* { dg-do compile } */
+-/* { dg-options "-O2" } */
+-
+-extern int *stderr;
+-
+-void f (int *, const char *, ...);
+-
+-void g (const char *conf_name)
+-{
+-  typedef struct
+-  {
+-    const char *label;
+-    const int value;
+-  } Section;
+-
+-  const Section sections[2] = { {"", 0}, {"", 1} };
+-
+-  f (stderr, "", "", conf_name, 0, sections[0]);
+-  f (stderr, "", "", conf_name, 0, sections[0]);
+-}
+diff --git a/gcc/testsuite/gcc.dg/pr40209.c b/gcc/testsuite/gcc.dg/pr40209.c
+index 4e77df5c2e6..c23d69d1f1b 100644
+--- a/gcc/testsuite/gcc.dg/pr40209.c
++++ b/gcc/testsuite/gcc.dg/pr40209.c
+@@ -1,6 +1,8 @@
+ /* { dg-do compile } */
+ /* { dg-options "-O2 -fprofile-use -fopt-info -Wno-missing-profile" } */
+ 
++
++
+ void process(const char *s);
+ 
+ struct BaseHolder {
+diff --git a/gcc/testsuite/gcc.dg/pr50209.c b/gcc/testsuite/gcc.dg/pr50209.c
+new file mode 100644
+index 00000000000..b28b04f6431
+--- /dev/null
++++ b/gcc/testsuite/gcc.dg/pr50209.c
+@@ -0,0 +1,3 @@
++
++
++
+diff --git a/gcc/testsuite/gcc.dg/pr63567-1.c b/gcc/testsuite/gcc.dg/pr63567-1.c
+index 97da171563e..00c5ecc11fa 100644
+--- a/gcc/testsuite/gcc.dg/pr63567-1.c
++++ b/gcc/testsuite/gcc.dg/pr63567-1.c
+@@ -1,3 +1,4 @@
++
+ /* PR c/63567 */
+ /* { dg-do compile } */
+ /* { dg-options "" } */
+diff --git a/gcc/varasm.c b/gcc/varasm.c
+index f062e48071f..fd3c7ca8cf3 100644
+--- a/gcc/varasm.c
++++ b/gcc/varasm.c
+@@ -1,3 +1,5 @@
++
++
+ /* Output variables, constants and external declarations, for GNU compiler.
+    Copyright (C) 1987-2020 Free Software Foundation, Inc.
+ 
+diff --git a/libssp/gets-chk.c b/libssp/gets-chk.c
+index 4ad78c1f77b..6687b368038 100644
+--- a/libssp/gets-chk.c
++++ b/libssp/gets-chk.c
+@@ -32,6 +32,8 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
+ <http://www.gnu.org/licenses/>.  */
+ 
+ 
++
++
+ #include "config.h"
+ #include <ssp/ssp.h>
+ #include <stdarg.h>
+'''
+
+EXPECTED4 = '''\
+gcc/ChangeLog:
+
+	* ipa-icf.c:
+	* varasm.c:
+
+libssp/ChangeLog:
+
+	* gets-chk.c:
+
+gcc/testsuite/ChangeLog:
+
+	* gcc.dg/pr40209.c:
+	* gcc.dg/pr63567-1.c:
+	* gcc.dg/pr32374.c: Removed.
+	* gcc.dg/pr50209.c: New test.
+
+'''
+
+class TestMklog(unittest.TestCase):
+    def test_macro_definition(self):
+        changelog = generate_changelog(PATCH1)
+        assert changelog == EXPECTED1
+
+    def test_changed_argument(self):
+        changelog = generate_changelog(PATCH2)
+        assert changelog == EXPECTED2
+
+    def test_enum_and_struct(self):
+        changelog = generate_changelog(PATCH3)
+        assert changelog == EXPECTED3
+
+    def test_no_function(self):
+        changelog = generate_changelog(PATCH3, True)
+        assert changelog == EXPECTED3B
+
+    def test_sorting(self):
+        changelog = generate_changelog(PATCH4)
+        assert changelog == EXPECTED4
diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c
index 069de9d82fb..aab79492357 100644
--- a/gcc/ipa-icf.c
+++ b/gcc/ipa-icf.c
@@ -1,3 +1,5 @@
+
+
 /* Interprocedural Identical Code Folding pass
    Copyright (C) 2014-2020 Free Software Foundation, Inc.
 
-- 
2.26.2
Thomas Rodgers via Gcc-patches May 19, 2020, 8:23 a.m. | #10
On Tue, May 19, 2020 at 10:11:28AM +0200, Martin Liška wrote:
> > I find this format more helpful for the reasons below so unless your

> > script can be tweaked to do something similar I'd like to be able to

> > continue to use mine going forward with the new infrastructure.

> 

> Let's extend the contrib script.


BTW, concerning mklog, the very common problem is that it doesn't do the
right thing because the patch doesn't contain enough context to figure out
what exactly has changed.  If the script would be used together with git
rather than just on a patch file, perhaps it could handle more, like
ask git for a patch with unlimited context (like -U100000000 on patch does).
The common problems I remember is that e.g. when changing a function comment
above some function, it is attributed to the previous function rather than
following, labels in function confusing it:
 void
 foo ()
 {
   ...
 label:
   ...
-  ...
+  ...
 }
will result in (label), GTY markers confusing it
 struct GTY foobar {
   ...
-  ...
+  ...
 };
resulting in (struct GTY) or so, another common problem is too large
function names (or more often *.md define_* names); here I'm afraid
diff doesn't have an argument to not truncate it, or sometimes e.g. changes
to #define being attributed to something else.
I know some of the issues can be pretty hard to deal with.

	Jakub
Martin Liška May 19, 2020, 8:53 a.m. | #11
On 5/19/20 10:11 AM, Martin Liška wrote:
> Can you please share how do you do it? It would be easy to add it.


I added the feature via --fill-up-bug-titles option. It uses common
request and beatifulsoup packages.

Martin
From 5450c99b54131d1942ece3ffb6bbe415b1c85151 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>

Date: Fri, 15 May 2020 00:44:07 +0200
Subject: [PATCH] New mklog script.

contrib/ChangeLog:

2020-05-15  Martin Liska  <mliska@suse.cz>

	* gcc-git-customization.sh: Add
	alias.gcc-mklog new hook.
	* mklog_ng.py: New file.
	* test_mklog_ng.py: New file.
---
 contrib/gcc-git-customization.sh |   2 +
 contrib/mklog_ng.py              | 223 ++++++++++++++++++++
 contrib/test_mklog_ng.py         | 345 +++++++++++++++++++++++++++++++
 gcc/ipa-icf.c                    |   2 +
 4 files changed, 572 insertions(+)
 create mode 100755 contrib/mklog_ng.py
 create mode 100755 contrib/test_mklog_ng.py

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index a932bf8c06a..b7b97327be3 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -25,6 +25,8 @@ git config alias.svn-rev '!f() { rev=$1; shift; git log --all --grep="^From-SVN:
 git config alias.gcc-descr \!"f() { if test \${1:-no} = --full; then c=\${2:-master}; r=\$(git describe --all --abbrev=40 --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-,r,p'); expr match \${r:-no} '^r[0-9]\\+\$' >/dev/null && r=\${r}-0-g\$(git rev-parse \${2:-master}); else c=\${1:-master}; r=\$(git describe --all --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)-\\([0-9]\\+\\)-g[0-9a-f]*\$,r\\2-\\3,p;s,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)\$,r\\2-0,p'); fi; if test -n \$r; then o=\$(git config --get gcc-config.upstream); rr=\$(echo \$r | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\\(-g[0-9a-f]\\+\\)\\?\$,\\1,p'); if git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$rr >/dev/null; then m=releases/gcc-\$rr; else m=master; fi; git merge-base --is-ancestor \$c \${o:-origin}/\$m && \echo \${r}; fi; }; f"
 git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream); r=\$(echo \$1 | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\$,\\1,p'); n=\$(echo \$1 | sed -n 's,^r[0-9]\\+-\\([0-9]\\+\\)\$,\\1,p'); test -z \$r && echo Invalid id \$1 && exit 1; h=\$(git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$r); test -z \$h && h=\$(git rev-parse --verify --quiet \${o:-origin}/master); p=\$(git describe --all --match 'basepoints/gcc-'\$r \$h | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+-\\([0-9]\\+\\)-g[0-9a-f]*\$,\\2,p;s,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+\$,0,p'); git rev-parse --verify \$h~\$(expr \$p - \$n); }; f"
 
+git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog_ng.py" $@; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
diff --git a/contrib/mklog_ng.py b/contrib/mklog_ng.py
new file mode 100755
index 00000000000..cc3f937c253
--- /dev/null
+++ b/contrib/mklog_ng.py
@@ -0,0 +1,223 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import argparse
+import bs4
+import os
+import re
+import requests
+import sys
+
+from unidiff import PatchSet
+
+pr_regex = re.compile(r'(\/(\/|\*)|[Cc*!])\s+(?P<pr>PR [a-z+-]+\/[0-9]+)')
+identifier_regex = re.compile(r'^([a-zA-Z0-9_#].*)')
+comment_regex = re.compile(r'^\/\*')
+struct_regex = re.compile(r'^((class|struct|union|enum)\s+[a-zA-Z0-9_]+)')
+macro_regex = re.compile(r'#\s*(define|undef)\s+([a-zA-Z0-9_]+)')
+super_macro_regex = re.compile(r'^DEF[A-Z0-9_]+\s*\(([a-zA-Z0-9_]+)')
+fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
+template_and_param_regex = re.compile(r'<[^<>]*>')
+
+function_extensions = set(['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def'])
+
+help_message = """\
+Generate ChangeLog template for PATCH.
+PATCH must be generated using diff(1)'s -up or -cp options
+(or their equivalent in git).
+"""
+
+script_folder = os.path.realpath(__file__)
+gcc_root = os.path.dirname(os.path.dirname(script_folder))
+
+
+def find_changelog(path):
+    folder = os.path.split(path)[0]
+    while True:
+        if os.path.exists(os.path.join(gcc_root, folder, 'ChangeLog')):
+            return folder
+        folder = os.path.dirname(folder)
+        if folder == '':
+            return folder
+    raise AssertionError()
+
+
+def extract_function_name(line):
+    if comment_regex.match(line):
+        return None
+    m = struct_regex.search(line)
+    if m:
+        # Struct declaration
+        return m.group(1)
+    m = macro_regex.search(line)
+    if m:
+        # Macro definition
+        return m.group(2)
+    m = super_macro_regex.search(line)
+    if m:
+        # Supermacro
+        return m.group(1)
+    m = fn_regex.search(line)
+    if m:
+        # Discard template and function parameters.
+        fn = m.group(1)
+        fn = re.sub(template_and_param_regex, '', fn)
+        return fn.rstrip()
+    return None
+
+
+def try_add_function(functions, line):
+    fn = extract_function_name(line)
+    if fn and fn not in functions:
+        functions.append(fn)
+    return bool(fn)
+
+
+def sort_changelog_files(changed_file):
+    return (changed_file.is_added_file, changed_file.is_removed_file)
+
+
+def get_pr_titles(prs):
+    if not prs:
+        return ''
+
+    output = ''
+    for pr in prs:
+        id = pr.split('/')[-1]
+        r = requests.get('https://gcc.gnu.org/PR%s' % id)
+        html = bs4.BeautifulSoup(r.text, features='lxml')
+        title = html.title.text
+        title = title[title.find('–') + 1:].strip()
+        output += '%s - %s\n' % (pr, title)
+    output += '\n'
+    return output
+
+def generate_changelog(data, no_functions=False, fill_pr_titles=False):
+    changelogs = {}
+    changelog_list = []
+    prs = []
+    out = ''
+    diff = PatchSet(data)
+
+    for file in diff:
+        changelog = find_changelog(file.path)
+        if changelog not in changelogs:
+            changelogs[changelog] = []
+            changelog_list.append(changelog)
+        changelogs[changelog].append(file)
+
+        # Extract PR entries from newly added tests
+        if 'testsuite' in file.path and file.is_added_file:
+            for line in list(file)[0]:
+                m = pr_regex.search(line.value)
+                if m:
+                    pr = m.group('pr')
+                    if pr not in prs:
+                        prs.append(pr)
+                else:
+                    break
+
+    if fill_pr_titles:
+        out += get_pr_titles(prs)
+
+    # sort ChangeLog so that 'testsuite' is at the end
+    for changelog in sorted(changelog_list, key=lambda x: 'testsuite' in x):
+        files = changelogs[changelog]
+        out += '%s:\n' % os.path.join(changelog, 'ChangeLog')
+        out += '\n'
+        for pr in prs:
+            out += '\t%s\n' % pr
+        # new and deleted files should be at the end
+        for file in sorted(files, key=sort_changelog_files):
+            assert file.path.startswith(changelog)
+            in_tests = 'testsuite' in changelog or 'testsuite' in file.path
+            relative_path = file.path[len(changelog):].lstrip('/')
+            functions = []
+            if file.is_added_file:
+                msg = 'New test' if in_tests else 'New file'
+                out += '\t* %s: %s.\n' % (relative_path, msg)
+            elif file.is_removed_file:
+                out += '\t* %s: Removed.\n' % (relative_path)
+            else:
+                if not no_functions:
+                    for hunk in file:
+                        # Do not add function names for testsuite files
+                        extension = os.path.splitext(relative_path)[1]
+                        if not in_tests and extension in function_extensions:
+                            last_fn = None
+                            modified_visited = False
+                            success = False
+                            for line in hunk:
+                                m = identifier_regex.match(line.value)
+                                if line.is_added or line.is_removed:
+                                    if not line.value.strip():
+                                        continue
+                                    modified_visited = True
+                                    if m and try_add_function(functions,
+                                                              m.group(1)):
+                                        last_fn = None
+                                        success = True
+                                elif line.is_context:
+                                    if last_fn and modified_visited:
+                                        try_add_function(functions, last_fn)
+                                        last_fn = None
+                                        modified_visited = False
+                                        success = True
+                                    elif m:
+                                        last_fn = m.group(1)
+                                        modified_visited = False
+                            if not success:
+                                try_add_function(functions,
+                                                 hunk.section_header)
+                if functions:
+                    out += '\t* %s (%s):\n' % (relative_path, functions[0])
+                    for fn in functions[1:]:
+                        out += '\t(%s):\n' % fn
+                else:
+                    out += '\t* %s:\n' % relative_path
+        out += '\n'
+    return out
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=help_message)
+    parser.add_argument('input', nargs='?',
+                        help='Patch file (or missing, read standard input)')
+    parser.add_argument('-s', '--no-functions', action='store_true',
+                        help='Do not generate function names in ChangeLogs')
+    parser.add_argument('-p', '--fill-up-bug-titles', action='store_true',
+                        help='Download title of mentioned PRs')
+    args = parser.parse_args()
+    if args.input == '-':
+        args.input = None
+
+    input = open(args.input) if args.input else sys.stdin
+    data = input.read()
+    output = generate_changelog(data, args.no_functions,
+                                args.fill_up_bug_titles)
+    print(output, end='')
diff --git a/contrib/test_mklog_ng.py b/contrib/test_mklog_ng.py
new file mode 100755
index 00000000000..7e948a17c6e
--- /dev/null
+++ b/contrib/test_mklog_ng.py
@@ -0,0 +1,345 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import unittest
+
+from mklog_ng import generate_changelog
+
+PATCH1 = '''\
+diff --git a/gcc/config/riscv/riscv.h b/gcc/config/riscv/riscv.h
+index 567c23380fe..e6209ede9d6 100644
+--- a/gcc/config/riscv/riscv.h
++++ b/gcc/config/riscv/riscv.h
+@@ -920,6 +920,7 @@ extern unsigned riscv_stack_boundary;
+ #define SHIFT_RS1 15
+ #define SHIFT_IMM 20
+ #define IMM_BITS 12
++#define C_S_BITS 5
+ #define C_SxSP_BITS 6
+ 
+ #define IMM_REACH (1LL << IMM_BITS)
+@@ -929,6 +930,10 @@ extern unsigned riscv_stack_boundary;
+ #define SWSP_REACH (4LL << C_SxSP_BITS)
+ #define SDSP_REACH (8LL << C_SxSP_BITS)
+ 
++/* This is the maximum value that can be represented in a compressed load/store
++   offset (an unsigned 5-bit value scaled by 4).  */
++#define CSW_MAX_OFFSET ((4LL << C_S_BITS) - 1) & ~3
++
+ /* Called from RISCV_REORG, this is defined in riscv-sr.c.  */
+ 
+ extern void riscv_remove_unneeded_save_restore_calls (void);
+
+'''
+
+EXPECTED1 = '''\
+gcc/ChangeLog:
+
+	* config/riscv/riscv.h (C_S_BITS):
+	(CSW_MAX_OFFSET):
+
+'''
+
+PATCH2 = '''\
+diff --git a/gcc/targhooks.h b/gcc/targhooks.h
+index 9704d23f1db..b572a36e8cf 100644
+--- a/gcc/targhooks.h
++++ b/gcc/targhooks.h
+@@ -120,7 +120,7 @@ extern bool default_empty_mask_is_expensive (unsigned);
+ extern void *default_init_cost (class loop *);
+ extern unsigned default_add_stmt_cost (class vec_info *, void *, int,
+ 				       enum vect_cost_for_stmt,
+-				       class _stmt_vec_info *, int,
++				       class _stmt_vec_info *, tree, int,
+ 				       enum vect_cost_model_location);
+ extern void default_finish_cost (void *, unsigned *, unsigned *, unsigned *);
+ extern void default_destroy_cost_data (void *);
+@@ -186,6 +186,7 @@ extern tree default_emutls_var_init (tree, tree, tree);
+ extern unsigned int default_hard_regno_nregs (unsigned int, machine_mode);
+ extern bool default_hard_regno_scratch_ok (unsigned int);
+ extern bool default_mode_dependent_address_p (const_rtx, addr_space_t);
++extern bool default_new_address_profitable_p (rtx, rtx_insn *, rtx);
+ extern bool default_target_option_valid_attribute_p (tree, tree, tree, int);
+ extern bool default_target_option_pragma_parse (tree, tree);
+ extern bool default_target_can_inline_p (tree, tree);
+
+'''
+
+EXPECTED2 = '''\
+gcc/ChangeLog:
+
+	* targhooks.h (default_add_stmt_cost):
+	(default_new_address_profitable_p):
+
+'''
+
+PATCH3 = '''\
+diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
+index 2b1e33f94ae..7f47402f9b9 100644
+--- a/libcpp/include/cpplib.h
++++ b/libcpp/include/cpplib.h
+@@ -173,7 +173,7 @@ enum c_lang {CLK_GNUC89 = 0, CLK_GNUC99, CLK_GNUC11, CLK_GNUC17, CLK_GNUC2X,
+ 	     CLK_STDC2X,
+ 	     CLK_GNUCXX, CLK_CXX98, CLK_GNUCXX11, CLK_CXX11,
+ 	     CLK_GNUCXX14, CLK_CXX14, CLK_GNUCXX17, CLK_CXX17,
+-	     CLK_GNUCXX2A, CLK_CXX2A, CLK_ASM};
++	     CLK_GNUCXX20, CLK_CXX20, CLK_ASM};
+ 
+ /* Payload of a NUMBER, STRING, CHAR or COMMENT token.  */
+ struct GTY(()) cpp_string {
+@@ -484,7 +484,7 @@ struct cpp_options
+   /* Nonzero for C2X decimal floating-point constants.  */
+   unsigned char dfp_constants;
+ 
+-  /* Nonzero for C++2a __VA_OPT__ feature.  */
++  /* Nonzero for C++20 __VA_OPT__ feature.  */
+   unsigned char va_opt;
+ 
+   /* Nonzero for the '::' token.  */
+
+'''
+
+EXPECTED3 = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h (enum c_lang):
+	(struct cpp_options):
+
+'''
+
+EXPECTED3B = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h:
+
+'''
+
+PATCH4 = '''\
+diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c
+index aab79492357..f0df1002488 100644
+--- a/gcc/ipa-icf.c
++++ b/gcc/ipa-icf.c
+@@ -1,5 +1,7 @@
+ 
+ 
++
++
+ /* Interprocedural Identical Code Folding pass
+    Copyright (C) 2014-2020 Free Software Foundation, Inc.
+ 
+diff --git a/gcc/testsuite/gcc.dg/pr32374.c b/gcc/testsuite/gcc.dg/pr32374.c
+deleted file mode 100644
+index de15d559f5b..00000000000
+--- a/gcc/testsuite/gcc.dg/pr32374.c
++++ /dev/null
+@@ -1,20 +0,0 @@
+-/* { dg-do compile } */
+-/* { dg-options "-O2" } */
+-
+-extern int *stderr;
+-
+-void f (int *, const char *, ...);
+-
+-void g (const char *conf_name)
+-{
+-  typedef struct
+-  {
+-    const char *label;
+-    const int value;
+-  } Section;
+-
+-  const Section sections[2] = { {"", 0}, {"", 1} };
+-
+-  f (stderr, "", "", conf_name, 0, sections[0]);
+-  f (stderr, "", "", conf_name, 0, sections[0]);
+-}
+diff --git a/gcc/testsuite/gcc.dg/pr40209.c b/gcc/testsuite/gcc.dg/pr40209.c
+index 4e77df5c2e6..c23d69d1f1b 100644
+--- a/gcc/testsuite/gcc.dg/pr40209.c
++++ b/gcc/testsuite/gcc.dg/pr40209.c
+@@ -1,6 +1,8 @@
+ /* { dg-do compile } */
+ /* { dg-options "-O2 -fprofile-use -fopt-info -Wno-missing-profile" } */
+ 
++
++
+ void process(const char *s);
+ 
+ struct BaseHolder {
+diff --git a/gcc/testsuite/gcc.dg/pr50209.c b/gcc/testsuite/gcc.dg/pr50209.c
+new file mode 100644
+index 00000000000..b28b04f6431
+--- /dev/null
++++ b/gcc/testsuite/gcc.dg/pr50209.c
+@@ -0,0 +1,3 @@
++
++
++
+diff --git a/gcc/testsuite/gcc.dg/pr63567-1.c b/gcc/testsuite/gcc.dg/pr63567-1.c
+index 97da171563e..00c5ecc11fa 100644
+--- a/gcc/testsuite/gcc.dg/pr63567-1.c
++++ b/gcc/testsuite/gcc.dg/pr63567-1.c
+@@ -1,3 +1,4 @@
++
+ /* PR c/63567 */
+ /* { dg-do compile } */
+ /* { dg-options "" } */
+diff --git a/gcc/varasm.c b/gcc/varasm.c
+index f062e48071f..fd3c7ca8cf3 100644
+--- a/gcc/varasm.c
++++ b/gcc/varasm.c
+@@ -1,3 +1,5 @@
++
++
+ /* Output variables, constants and external declarations, for GNU compiler.
+    Copyright (C) 1987-2020 Free Software Foundation, Inc.
+ 
+diff --git a/libssp/gets-chk.c b/libssp/gets-chk.c
+index 4ad78c1f77b..6687b368038 100644
+--- a/libssp/gets-chk.c
++++ b/libssp/gets-chk.c
+@@ -32,6 +32,8 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
+ <http://www.gnu.org/licenses/>.  */
+ 
+ 
++
++
+ #include "config.h"
+ #include <ssp/ssp.h>
+ #include <stdarg.h>
+'''
+
+EXPECTED4 = '''\
+gcc/ChangeLog:
+
+	* ipa-icf.c:
+	* varasm.c:
+
+libssp/ChangeLog:
+
+	* gets-chk.c:
+
+gcc/testsuite/ChangeLog:
+
+	* gcc.dg/pr40209.c:
+	* gcc.dg/pr63567-1.c:
+	* gcc.dg/pr32374.c: Removed.
+	* gcc.dg/pr50209.c: New test.
+
+'''
+
+PATCH5 = '''\
+diff --git a/gcc/testsuite/gcc.target/i386/pr95046-6.c b/gcc/testsuite/gcc.target/i386/pr95046-6.c
+new file mode 100644
+index 00000000000..dcc8999c446
+--- /dev/null
++++ b/gcc/testsuite/gcc.target/i386/pr95046-6.c
+@@ -0,0 +1,44 @@
++/* PR target/95046 */
++/* { dg-do compile { target { ! ia32 } } } */
++/* { dg-options "-O3 -mavx512vl" } */
++
++
++double r[2];
++int s[2];
++unsigned int u[2];
++
++void
++test_float (void)
++{
++  for (int i = 0; i < 2; i++)
++    r[i] = s[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvtdq2pd" } } */
++
++void
++test_ufloat (void)
++{
++  for (int i = 0; i < 2; i++)
++    r[i] = u[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvtudq2pd" } } */
++
++void
++test_fix (void)
++{
++  for (int i = 0; i < 2; i++)
++    s[i] = r[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvttpd2dqx" } } */
++
++void
++test_ufix (void)
++{
++  for (int i = 0; i < 2; i++)
++    u[i] = r[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvttpd2udqx" } } */
+-- 
+2.26.2
+
+'''
+
+EXPECTED5 = '''\
+PR target/95046 - Vectorize V2SFmode operations
+
+gcc/testsuite/ChangeLog:
+
+	PR target/95046
+	* gcc.target/i386/pr95046-6.c: New test.
+
+'''
+
+class TestMklog(unittest.TestCase):
+    def test_macro_definition(self):
+        changelog = generate_changelog(PATCH1)
+        assert changelog == EXPECTED1
+
+    def test_changed_argument(self):
+        changelog = generate_changelog(PATCH2)
+        assert changelog == EXPECTED2
+
+    def test_enum_and_struct(self):
+        changelog = generate_changelog(PATCH3)
+        assert changelog == EXPECTED3
+
+    def test_no_function(self):
+        changelog = generate_changelog(PATCH3, True)
+        assert changelog == EXPECTED3B
+
+    def test_sorting(self):
+        changelog = generate_changelog(PATCH4)
+        assert changelog == EXPECTED4
+
+    def test_pr_bugzilla_download(self):
+        changelog = generate_changelog(PATCH5, fill_pr_titles=True)
+        assert changelog == EXPECTED5
diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c
index 069de9d82fb..aab79492357 100644
--- a/gcc/ipa-icf.c
+++ b/gcc/ipa-icf.c
@@ -1,3 +1,5 @@
+
+
 /* Interprocedural Identical Code Folding pass
    Copyright (C) 2014-2020 Free Software Foundation, Inc.
 
-- 
2.26.2
Martin Liška May 19, 2020, 8:55 a.m. | #12
On 5/19/20 10:23 AM, Jakub Jelinek wrote:
> On Tue, May 19, 2020 at 10:11:28AM +0200, Martin Liška wrote:

>>> I find this format more helpful for the reasons below so unless your

>>> script can be tweaked to do something similar I'd like to be able to

>>> continue to use mine going forward with the new infrastructure.

>>

>> Let's extend the contrib script.

> 

> BTW, concerning mklog, the very common problem is that it doesn't do the

> right thing because the patch doesn't contain enough context to figure out

> what exactly has changed.  If the script would be used together with git

> rather than just on a patch file, perhaps it could handle more, like

> ask git for a patch with unlimited context (like -U100000000 on patch does).


Good idea but the regex parsing takes some time.

> The common problems I remember is that e.g. when changing a function comment

> above some function, it is attributed to the previous function rather than

> following, labels in function confusing it:

>   void

>   foo ()

>   {

>     ...

>   label:

>     ...

> -  ...

> +  ...

>   }


I've just tested that and it will take function for patch context (sem_variable::equals):
@@ -1875,6 +1875,7 @@ sem_variable::equals (tree t1, tree t2)
      default:
        return return_false_with_msg ("Unknown TREE code reached");
      }
+
  }

> will result in (label), GTY markers confusing it

>   struct GTY foobar {

>     ...

> -  ...

> +  ...

>   };

> resulting in (struct GTY)


Yes, I know about these and I'll improve stripping of GTY markers.

> or so, another common problem is too large

> function names (or more often *.md define_* names); here I'm afraid

> diff doesn't have an argument to not truncate it, or sometimes e.g. changes

> to #define being attributed to something else.

> I know some of the issues can be pretty hard to deal with.


;)

Martin

> 

> 	Jakub

>
Martin Liška May 19, 2020, 9:38 a.m. | #13
On 5/19/20 10:53 AM, Martin Liška wrote:
> On 5/19/20 10:11 AM, Martin Liška wrote:

>> Can you please share how do you do it? It would be easy to add it.

> 

> I added the feature via --fill-up-bug-titles option. It uses common

> request and beatifulsoup packages.

> 

> Martin


Ok, I'm going to install the following 2 patches that put 2 legacy scripts
into contrib/legacy folder. And a new 'git gcc-mklog' alias is added for the
new one.

Hope other are fine with the change. I'm planning to work on the new script
and fix future limitations.

Martin
From 577083b84f6851eee0b2d36e26aef42f69676942 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>

Date: Fri, 15 May 2020 00:44:07 +0200
Subject: [PATCH 2/2] New mklog script.

contrib/ChangeLog:

2020-05-15  Martin Liska  <mliska@suse.cz>

	* gcc-git-customization.sh: Add
	alias.gcc-mklog new hook.
	* mklog.py: New file.
	* test_mklog.py: New file.
---
 contrib/gcc-git-customization.sh |   2 +
 contrib/mklog.py                 | 223 ++++++++++++++++++++
 contrib/test_mklog.py            | 345 +++++++++++++++++++++++++++++++
 3 files changed, 570 insertions(+)
 create mode 100755 contrib/mklog.py
 create mode 100755 contrib/test_mklog.py

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index ce293d1fe42..91d378ba32a 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -27,6 +27,8 @@ git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream)
 
 git config alias.gcc-verify '!f() { "`git rev-parse --show-toplevel`/contrib/gcc-changelog/git_check_commit.py" $@; } ; f'
 
+git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog.py" $@; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
diff --git a/contrib/mklog.py b/contrib/mklog.py
new file mode 100755
index 00000000000..cc3f937c253
--- /dev/null
+++ b/contrib/mklog.py
@@ -0,0 +1,223 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import argparse
+import bs4
+import os
+import re
+import requests
+import sys
+
+from unidiff import PatchSet
+
+pr_regex = re.compile(r'(\/(\/|\*)|[Cc*!])\s+(?P<pr>PR [a-z+-]+\/[0-9]+)')
+identifier_regex = re.compile(r'^([a-zA-Z0-9_#].*)')
+comment_regex = re.compile(r'^\/\*')
+struct_regex = re.compile(r'^((class|struct|union|enum)\s+[a-zA-Z0-9_]+)')
+macro_regex = re.compile(r'#\s*(define|undef)\s+([a-zA-Z0-9_]+)')
+super_macro_regex = re.compile(r'^DEF[A-Z0-9_]+\s*\(([a-zA-Z0-9_]+)')
+fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
+template_and_param_regex = re.compile(r'<[^<>]*>')
+
+function_extensions = set(['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def'])
+
+help_message = """\
+Generate ChangeLog template for PATCH.
+PATCH must be generated using diff(1)'s -up or -cp options
+(or their equivalent in git).
+"""
+
+script_folder = os.path.realpath(__file__)
+gcc_root = os.path.dirname(os.path.dirname(script_folder))
+
+
+def find_changelog(path):
+    folder = os.path.split(path)[0]
+    while True:
+        if os.path.exists(os.path.join(gcc_root, folder, 'ChangeLog')):
+            return folder
+        folder = os.path.dirname(folder)
+        if folder == '':
+            return folder
+    raise AssertionError()
+
+
+def extract_function_name(line):
+    if comment_regex.match(line):
+        return None
+    m = struct_regex.search(line)
+    if m:
+        # Struct declaration
+        return m.group(1)
+    m = macro_regex.search(line)
+    if m:
+        # Macro definition
+        return m.group(2)
+    m = super_macro_regex.search(line)
+    if m:
+        # Supermacro
+        return m.group(1)
+    m = fn_regex.search(line)
+    if m:
+        # Discard template and function parameters.
+        fn = m.group(1)
+        fn = re.sub(template_and_param_regex, '', fn)
+        return fn.rstrip()
+    return None
+
+
+def try_add_function(functions, line):
+    fn = extract_function_name(line)
+    if fn and fn not in functions:
+        functions.append(fn)
+    return bool(fn)
+
+
+def sort_changelog_files(changed_file):
+    return (changed_file.is_added_file, changed_file.is_removed_file)
+
+
+def get_pr_titles(prs):
+    if not prs:
+        return ''
+
+    output = ''
+    for pr in prs:
+        id = pr.split('/')[-1]
+        r = requests.get('https://gcc.gnu.org/PR%s' % id)
+        html = bs4.BeautifulSoup(r.text, features='lxml')
+        title = html.title.text
+        title = title[title.find('–') + 1:].strip()
+        output += '%s - %s\n' % (pr, title)
+    output += '\n'
+    return output
+
+def generate_changelog(data, no_functions=False, fill_pr_titles=False):
+    changelogs = {}
+    changelog_list = []
+    prs = []
+    out = ''
+    diff = PatchSet(data)
+
+    for file in diff:
+        changelog = find_changelog(file.path)
+        if changelog not in changelogs:
+            changelogs[changelog] = []
+            changelog_list.append(changelog)
+        changelogs[changelog].append(file)
+
+        # Extract PR entries from newly added tests
+        if 'testsuite' in file.path and file.is_added_file:
+            for line in list(file)[0]:
+                m = pr_regex.search(line.value)
+                if m:
+                    pr = m.group('pr')
+                    if pr not in prs:
+                        prs.append(pr)
+                else:
+                    break
+
+    if fill_pr_titles:
+        out += get_pr_titles(prs)
+
+    # sort ChangeLog so that 'testsuite' is at the end
+    for changelog in sorted(changelog_list, key=lambda x: 'testsuite' in x):
+        files = changelogs[changelog]
+        out += '%s:\n' % os.path.join(changelog, 'ChangeLog')
+        out += '\n'
+        for pr in prs:
+            out += '\t%s\n' % pr
+        # new and deleted files should be at the end
+        for file in sorted(files, key=sort_changelog_files):
+            assert file.path.startswith(changelog)
+            in_tests = 'testsuite' in changelog or 'testsuite' in file.path
+            relative_path = file.path[len(changelog):].lstrip('/')
+            functions = []
+            if file.is_added_file:
+                msg = 'New test' if in_tests else 'New file'
+                out += '\t* %s: %s.\n' % (relative_path, msg)
+            elif file.is_removed_file:
+                out += '\t* %s: Removed.\n' % (relative_path)
+            else:
+                if not no_functions:
+                    for hunk in file:
+                        # Do not add function names for testsuite files
+                        extension = os.path.splitext(relative_path)[1]
+                        if not in_tests and extension in function_extensions:
+                            last_fn = None
+                            modified_visited = False
+                            success = False
+                            for line in hunk:
+                                m = identifier_regex.match(line.value)
+                                if line.is_added or line.is_removed:
+                                    if not line.value.strip():
+                                        continue
+                                    modified_visited = True
+                                    if m and try_add_function(functions,
+                                                              m.group(1)):
+                                        last_fn = None
+                                        success = True
+                                elif line.is_context:
+                                    if last_fn and modified_visited:
+                                        try_add_function(functions, last_fn)
+                                        last_fn = None
+                                        modified_visited = False
+                                        success = True
+                                    elif m:
+                                        last_fn = m.group(1)
+                                        modified_visited = False
+                            if not success:
+                                try_add_function(functions,
+                                                 hunk.section_header)
+                if functions:
+                    out += '\t* %s (%s):\n' % (relative_path, functions[0])
+                    for fn in functions[1:]:
+                        out += '\t(%s):\n' % fn
+                else:
+                    out += '\t* %s:\n' % relative_path
+        out += '\n'
+    return out
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=help_message)
+    parser.add_argument('input', nargs='?',
+                        help='Patch file (or missing, read standard input)')
+    parser.add_argument('-s', '--no-functions', action='store_true',
+                        help='Do not generate function names in ChangeLogs')
+    parser.add_argument('-p', '--fill-up-bug-titles', action='store_true',
+                        help='Download title of mentioned PRs')
+    args = parser.parse_args()
+    if args.input == '-':
+        args.input = None
+
+    input = open(args.input) if args.input else sys.stdin
+    data = input.read()
+    output = generate_changelog(data, args.no_functions,
+                                args.fill_up_bug_titles)
+    print(output, end='')
diff --git a/contrib/test_mklog.py b/contrib/test_mklog.py
new file mode 100755
index 00000000000..ca7b9e79d95
--- /dev/null
+++ b/contrib/test_mklog.py
@@ -0,0 +1,345 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import unittest
+
+from mklog import generate_changelog
+
+PATCH1 = '''\
+diff --git a/gcc/config/riscv/riscv.h b/gcc/config/riscv/riscv.h
+index 567c23380fe..e6209ede9d6 100644
+--- a/gcc/config/riscv/riscv.h
++++ b/gcc/config/riscv/riscv.h
+@@ -920,6 +920,7 @@ extern unsigned riscv_stack_boundary;
+ #define SHIFT_RS1 15
+ #define SHIFT_IMM 20
+ #define IMM_BITS 12
++#define C_S_BITS 5
+ #define C_SxSP_BITS 6
+ 
+ #define IMM_REACH (1LL << IMM_BITS)
+@@ -929,6 +930,10 @@ extern unsigned riscv_stack_boundary;
+ #define SWSP_REACH (4LL << C_SxSP_BITS)
+ #define SDSP_REACH (8LL << C_SxSP_BITS)
+ 
++/* This is the maximum value that can be represented in a compressed load/store
++   offset (an unsigned 5-bit value scaled by 4).  */
++#define CSW_MAX_OFFSET ((4LL << C_S_BITS) - 1) & ~3
++
+ /* Called from RISCV_REORG, this is defined in riscv-sr.c.  */
+ 
+ extern void riscv_remove_unneeded_save_restore_calls (void);
+
+'''
+
+EXPECTED1 = '''\
+gcc/ChangeLog:
+
+	* config/riscv/riscv.h (C_S_BITS):
+	(CSW_MAX_OFFSET):
+
+'''
+
+PATCH2 = '''\
+diff --git a/gcc/targhooks.h b/gcc/targhooks.h
+index 9704d23f1db..b572a36e8cf 100644
+--- a/gcc/targhooks.h
++++ b/gcc/targhooks.h
+@@ -120,7 +120,7 @@ extern bool default_empty_mask_is_expensive (unsigned);
+ extern void *default_init_cost (class loop *);
+ extern unsigned default_add_stmt_cost (class vec_info *, void *, int,
+ 				       enum vect_cost_for_stmt,
+-				       class _stmt_vec_info *, int,
++				       class _stmt_vec_info *, tree, int,
+ 				       enum vect_cost_model_location);
+ extern void default_finish_cost (void *, unsigned *, unsigned *, unsigned *);
+ extern void default_destroy_cost_data (void *);
+@@ -186,6 +186,7 @@ extern tree default_emutls_var_init (tree, tree, tree);
+ extern unsigned int default_hard_regno_nregs (unsigned int, machine_mode);
+ extern bool default_hard_regno_scratch_ok (unsigned int);
+ extern bool default_mode_dependent_address_p (const_rtx, addr_space_t);
++extern bool default_new_address_profitable_p (rtx, rtx_insn *, rtx);
+ extern bool default_target_option_valid_attribute_p (tree, tree, tree, int);
+ extern bool default_target_option_pragma_parse (tree, tree);
+ extern bool default_target_can_inline_p (tree, tree);
+
+'''
+
+EXPECTED2 = '''\
+gcc/ChangeLog:
+
+	* targhooks.h (default_add_stmt_cost):
+	(default_new_address_profitable_p):
+
+'''
+
+PATCH3 = '''\
+diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
+index 2b1e33f94ae..7f47402f9b9 100644
+--- a/libcpp/include/cpplib.h
++++ b/libcpp/include/cpplib.h
+@@ -173,7 +173,7 @@ enum c_lang {CLK_GNUC89 = 0, CLK_GNUC99, CLK_GNUC11, CLK_GNUC17, CLK_GNUC2X,
+ 	     CLK_STDC2X,
+ 	     CLK_GNUCXX, CLK_CXX98, CLK_GNUCXX11, CLK_CXX11,
+ 	     CLK_GNUCXX14, CLK_CXX14, CLK_GNUCXX17, CLK_CXX17,
+-	     CLK_GNUCXX2A, CLK_CXX2A, CLK_ASM};
++	     CLK_GNUCXX20, CLK_CXX20, CLK_ASM};
+ 
+ /* Payload of a NUMBER, STRING, CHAR or COMMENT token.  */
+ struct GTY(()) cpp_string {
+@@ -484,7 +484,7 @@ struct cpp_options
+   /* Nonzero for C2X decimal floating-point constants.  */
+   unsigned char dfp_constants;
+ 
+-  /* Nonzero for C++2a __VA_OPT__ feature.  */
++  /* Nonzero for C++20 __VA_OPT__ feature.  */
+   unsigned char va_opt;
+ 
+   /* Nonzero for the '::' token.  */
+
+'''
+
+EXPECTED3 = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h (enum c_lang):
+	(struct cpp_options):
+
+'''
+
+EXPECTED3B = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h:
+
+'''
+
+PATCH4 = '''\
+diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c
+index aab79492357..f0df1002488 100644
+--- a/gcc/ipa-icf.c
++++ b/gcc/ipa-icf.c
+@@ -1,5 +1,7 @@
+ 
+ 
++
++
+ /* Interprocedural Identical Code Folding pass
+    Copyright (C) 2014-2020 Free Software Foundation, Inc.
+ 
+diff --git a/gcc/testsuite/gcc.dg/pr32374.c b/gcc/testsuite/gcc.dg/pr32374.c
+deleted file mode 100644
+index de15d559f5b..00000000000
+--- a/gcc/testsuite/gcc.dg/pr32374.c
++++ /dev/null
+@@ -1,20 +0,0 @@
+-/* { dg-do compile } */
+-/* { dg-options "-O2" } */
+-
+-extern int *stderr;
+-
+-void f (int *, const char *, ...);
+-
+-void g (const char *conf_name)
+-{
+-  typedef struct
+-  {
+-    const char *label;
+-    const int value;
+-  } Section;
+-
+-  const Section sections[2] = { {"", 0}, {"", 1} };
+-
+-  f (stderr, "", "", conf_name, 0, sections[0]);
+-  f (stderr, "", "", conf_name, 0, sections[0]);
+-}
+diff --git a/gcc/testsuite/gcc.dg/pr40209.c b/gcc/testsuite/gcc.dg/pr40209.c
+index 4e77df5c2e6..c23d69d1f1b 100644
+--- a/gcc/testsuite/gcc.dg/pr40209.c
++++ b/gcc/testsuite/gcc.dg/pr40209.c
+@@ -1,6 +1,8 @@
+ /* { dg-do compile } */
+ /* { dg-options "-O2 -fprofile-use -fopt-info -Wno-missing-profile" } */
+ 
++
++
+ void process(const char *s);
+ 
+ struct BaseHolder {
+diff --git a/gcc/testsuite/gcc.dg/pr50209.c b/gcc/testsuite/gcc.dg/pr50209.c
+new file mode 100644
+index 00000000000..b28b04f6431
+--- /dev/null
++++ b/gcc/testsuite/gcc.dg/pr50209.c
+@@ -0,0 +1,3 @@
++
++
++
+diff --git a/gcc/testsuite/gcc.dg/pr63567-1.c b/gcc/testsuite/gcc.dg/pr63567-1.c
+index 97da171563e..00c5ecc11fa 100644
+--- a/gcc/testsuite/gcc.dg/pr63567-1.c
++++ b/gcc/testsuite/gcc.dg/pr63567-1.c
+@@ -1,3 +1,4 @@
++
+ /* PR c/63567 */
+ /* { dg-do compile } */
+ /* { dg-options "" } */
+diff --git a/gcc/varasm.c b/gcc/varasm.c
+index f062e48071f..fd3c7ca8cf3 100644
+--- a/gcc/varasm.c
++++ b/gcc/varasm.c
+@@ -1,3 +1,5 @@
++
++
+ /* Output variables, constants and external declarations, for GNU compiler.
+    Copyright (C) 1987-2020 Free Software Foundation, Inc.
+ 
+diff --git a/libssp/gets-chk.c b/libssp/gets-chk.c
+index 4ad78c1f77b..6687b368038 100644
+--- a/libssp/gets-chk.c
++++ b/libssp/gets-chk.c
+@@ -32,6 +32,8 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
+ <http://www.gnu.org/licenses/>.  */
+ 
+ 
++
++
+ #include "config.h"
+ #include <ssp/ssp.h>
+ #include <stdarg.h>
+'''
+
+EXPECTED4 = '''\
+gcc/ChangeLog:
+
+	* ipa-icf.c:
+	* varasm.c:
+
+libssp/ChangeLog:
+
+	* gets-chk.c:
+
+gcc/testsuite/ChangeLog:
+
+	* gcc.dg/pr40209.c:
+	* gcc.dg/pr63567-1.c:
+	* gcc.dg/pr32374.c: Removed.
+	* gcc.dg/pr50209.c: New test.
+
+'''
+
+PATCH5 = '''\
+diff --git a/gcc/testsuite/gcc.target/i386/pr95046-6.c b/gcc/testsuite/gcc.target/i386/pr95046-6.c
+new file mode 100644
+index 00000000000..dcc8999c446
+--- /dev/null
++++ b/gcc/testsuite/gcc.target/i386/pr95046-6.c
+@@ -0,0 +1,44 @@
++/* PR target/95046 */
++/* { dg-do compile { target { ! ia32 } } } */
++/* { dg-options "-O3 -mavx512vl" } */
++
++
++double r[2];
++int s[2];
++unsigned int u[2];
++
++void
++test_float (void)
++{
++  for (int i = 0; i < 2; i++)
++    r[i] = s[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvtdq2pd" } } */
++
++void
++test_ufloat (void)
++{
++  for (int i = 0; i < 2; i++)
++    r[i] = u[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvtudq2pd" } } */
++
++void
++test_fix (void)
++{
++  for (int i = 0; i < 2; i++)
++    s[i] = r[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvttpd2dqx" } } */
++
++void
++test_ufix (void)
++{
++  for (int i = 0; i < 2; i++)
++    u[i] = r[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvttpd2udqx" } } */
+-- 
+2.26.2
+
+'''
+
+EXPECTED5 = '''\
+PR target/95046 - Vectorize V2SFmode operations
+
+gcc/testsuite/ChangeLog:
+
+	PR target/95046
+	* gcc.target/i386/pr95046-6.c: New test.
+
+'''
+
+class TestMklog(unittest.TestCase):
+    def test_macro_definition(self):
+        changelog = generate_changelog(PATCH1)
+        assert changelog == EXPECTED1
+
+    def test_changed_argument(self):
+        changelog = generate_changelog(PATCH2)
+        assert changelog == EXPECTED2
+
+    def test_enum_and_struct(self):
+        changelog = generate_changelog(PATCH3)
+        assert changelog == EXPECTED3
+
+    def test_no_function(self):
+        changelog = generate_changelog(PATCH3, True)
+        assert changelog == EXPECTED3B
+
+    def test_sorting(self):
+        changelog = generate_changelog(PATCH4)
+        assert changelog == EXPECTED4
+
+    def test_pr_bugzilla_download(self):
+        changelog = generate_changelog(PATCH5, fill_pr_titles=True)
+        assert changelog == EXPECTED5
-- 
2.26.2
From 28bbebabc1d4849555ef62d79de034850a4f069d Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>

Date: Tue, 19 May 2020 11:02:03 +0200
Subject: [PATCH 1/2] Move 2 mklog scripts to legacy subfolder.

contrib/ChangeLog:

	* legacy/mklog: Moved from mklog.
	* legacy/mklog.pl: Moved from mklog.pl.
---
 contrib/{ => legacy}/mklog    | 0
 contrib/{ => legacy}/mklog.pl | 0
 2 files changed, 0 insertions(+), 0 deletions(-)
 rename contrib/{ => legacy}/mklog (100%)
 rename contrib/{ => legacy}/mklog.pl (100%)

diff --git a/contrib/mklog b/contrib/legacy/mklog
similarity index 100%
rename from contrib/mklog
rename to contrib/legacy/mklog
diff --git a/contrib/mklog.pl b/contrib/legacy/mklog.pl
similarity index 100%
rename from contrib/mklog.pl
rename to contrib/legacy/mklog.pl
-- 
2.26.2
Michael Matz May 19, 2020, 2:51 p.m. | #14
Hello,

On Tue, 19 May 2020, Martin Liška wrote:

> > The common problems I remember is that e.g. when changing a function comment

> > above some function, it is attributed to the previous function rather than

> > following, labels in function confusing it:

> >   void

> >   foo ()

> >   {

> >     ...

> >   label:

> >     ...

> > -  ...

> > +  ...

> >   }

> 

> I've just tested that and it will take function for patch context

> (sem_variable::equals):

> @@ -1875,6 +1875,7 @@ sem_variable::equals (tree t1, tree t2)

>      default:

>        return return_false_with_msg ("Unknown TREE code reached");

>      }

> +

>  }


No, the problem happens when the label is at column 0, like function names 
are.  Basically diff -p uses a regexp morally equivalent to 
'^[[:alpha:]$_]' to detect function headers, and git diff -p and friends 
followed suit.  But it should use something like
'^[[:alpha:]$_].*[^:]$' to rule out things ending with ':'.  See also diff 
-F for GNU diff.


Ciao,
Michael.
Joseph Myers May 19, 2020, 3:53 p.m. | #15
On Tue, 19 May 2020, Martin Liška wrote:

> On 5/19/20 10:11 AM, Martin Liška wrote:

> > Can you please share how do you do it? It would be easy to add it.

> 

> I added the feature via --fill-up-bug-titles option. It uses common

> request and beatifulsoup packages.


The REST interface is much better to use for extracting bug data than 
screen scraping of HTML output.  Fetch e.g. 
https://gcc.gnu.org/bugzilla/rest.cgi/bug?id=12345&include_fields=summary 
to get JSON bug data (change or omit include_fields if you want more than 
just the summary).

-- 
Joseph S. Myers
joseph@codesourcery.com
Richard Earnshaw May 19, 2020, 4:21 p.m. | #16
On 19/05/2020 15:51, Michael Matz wrote:
> Hello,

> 

> On Tue, 19 May 2020, Martin Liška wrote:

> 

>>> The common problems I remember is that e.g. when changing a function comment

>>> above some function, it is attributed to the previous function rather than

>>> following, labels in function confusing it:

>>>   void

>>>   foo ()

>>>   {

>>>     ...

>>>   label:

>>>     ...

>>> -  ...

>>> +  ...

>>>   }

>>

>> I've just tested that and it will take function for patch context

>> (sem_variable::equals):

>> @@ -1875,6 +1875,7 @@ sem_variable::equals (tree t1, tree t2)

>>      default:

>>        return return_false_with_msg ("Unknown TREE code reached");

>>      }

>> +

>>  }

> 

> No, the problem happens when the label is at column 0, like function names 

> are.  Basically diff -p uses a regexp morally equivalent to 

> '^[[:alpha:]$_]' to detect function headers, and git diff -p and friends 

> followed suit.  But it should use something like

> '^[[:alpha:]$_].*[^:]$' to rule out things ending with ':'.  See also diff 

> -F for GNU diff.

> 

> 

> Ciao,

> Michael.

> 


This is really a wart in the GNU coding style.  And one reason why I
tend to indent such labels by a single space.  It particularly affects
things like class definitions where public, private, etc statements
often appear in column 0.

IMO, it would be nice to get an official change in the coding style for
this, it's really irritating.

R.
Thomas Rodgers via Gcc-patches May 19, 2020, 4:34 p.m. | #17
On Tue, May 19, 2020 at 05:21:16PM +0100, Richard Earnshaw wrote:
> This is really a wart in the GNU coding style.  And one reason why I

> tend to indent such labels by a single space.  It particularly affects

> things like class definitions where public, private, etc statements

> often appear in column 0.

> 

> IMO, it would be nice to get an official change in the coding style for

> this, it's really irritating.


It doesn't have to be just label,
void
foo ()
{
  ...
#define X ...
  ...
#undef X
  ...
}
does the similar thing for mklog.

	Jakub
Martin Liška May 19, 2020, 7:15 p.m. | #18
On 5/19/20 5:53 PM, Joseph Myers wrote:
> On Tue, 19 May 2020, Martin Liška wrote:

> 

>> On 5/19/20 10:11 AM, Martin Liška wrote:

>>> Can you please share how do you do it? It would be easy to add it.

>>

>> I added the feature via --fill-up-bug-titles option. It uses common

>> request and beatifulsoup packages.

> 

> The REST interface is much better to use for extracting bug data than

> screen scraping of HTML output.  Fetch e.g.

> https://gcc.gnu.org/bugzilla/rest.cgi/bug?id=12345&include_fields=summary

> to get JSON bug data (change or omit include_fields if you want more than

> just the summary).

> 


You are right, there's a patch I'm going to install.

Martin
From b5a89069a074aff0ae94176c676eda069ff0a1c3 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>

Date: Tue, 19 May 2020 21:14:36 +0200
Subject: [PATCH] Use REST API for bug titles in mklog.

contrib/ChangeLog:

2020-05-19  Martin Liska  <mliska@suse.cz>

	* mklog.py: Use REST API for bug title downloading.
---
 contrib/mklog.py | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/contrib/mklog.py b/contrib/mklog.py
index 45559afbe6b..b27fad0ca2e 100755
--- a/contrib/mklog.py
+++ b/contrib/mklog.py
@@ -31,8 +31,6 @@ import os
 import re
 import sys
 
-import bs4
-
 import requests
 
 from unidiff import PatchSet
@@ -46,6 +44,8 @@ macro_regex = re.compile(r'#\s*(define|undef)\s+([a-zA-Z0-9_]+)')
 super_macro_regex = re.compile(r'^DEF[A-Z0-9_]+\s*\(([a-zA-Z0-9_]+)')
 fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
 template_and_param_regex = re.compile(r'<[^<>]*>')
+bugzilla_url = 'https://gcc.gnu.org/bugzilla/rest.cgi/bug?id=%s&' \
+               'include_fields=summary'
 
 function_extensions = set(['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def'])
 
@@ -106,18 +106,16 @@ def sort_changelog_files(changed_file):
 
 
 def get_pr_titles(prs):
-    if not prs:
-        return ''
-
     output = ''
     for pr in prs:
         id = pr.split('/')[-1]
-        r = requests.get('https://gcc.gnu.org/PR%s' % id)
-        html = bs4.BeautifulSoup(r.text, features='lxml')
-        title = html.title.text
-        title = title[title.find('–') + 1:].strip()
-        output += '%s - %s\n' % (pr, title)
-    output += '\n'
+        r = requests.get(bugzilla_url % id)
+        bugs = r.json()['bugs']
+        if len(bugs) == 1:
+            output += '%s - %s\n' % (pr, bugs[0]['summary'])
+            print(output)
+    if output:
+        output += '\n'
     return output
 
 
-- 
2.26.2
Thomas Rodgers via Gcc-patches May 19, 2020, 9:50 p.m. | #19
On Tue, 19 May 2020 at 17:13, Joseph Myers wrote:
>

> On Tue, 19 May 2020, Martin Liška wrote:

>

> > On 5/19/20 10:11 AM, Martin Liška wrote:

> > > Can you please share how do you do it? It would be easy to add it.

> >

> > I added the feature via --fill-up-bug-titles option. It uses common

> > request and beatifulsoup packages.

>

> The REST interface is much better to use for extracting bug data than

> screen scraping of HTML output.  Fetch e.g.

> https://gcc.gnu.org/bugzilla/rest.cgi/bug?id=12345&include_fields=summary

> to get JSON bug data (change or omit include_fields if you want more than

> just the summary).


REST+JSON is probably better for the mklog.py script, but in case this
is useful to anybody else, I get bugzilla info by fetching the XML for
a bug using the URL
https://gcc.gnu.org/bugzilla/show_bug.cgi?ctype=xml&id=NNNN and
applying this stylesheet:

    <?xml version="1.0" encoding="utf-8"?>
    <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
version="1.0">
    <xsl:output method="text"/>
    <xsl:template match="/">
      <xsl:apply-templates select="/bugzilla/bug/short_desc"/>
      <xsl:text> </xsl:text>
      <xsl:apply-templates select="/bugzilla/bug/bug_status"/>
      <xsl:text> </xsl:text>
      <xsl:apply-templates select="/bugzilla/bug/resolution"/>
      <xsl:text>
    </xsl:text>
    </xsl:template>
    </xsl:stylesheet>
Thomas Rodgers via Gcc-patches May 19, 2020, 9:54 p.m. | #20
On Tue, 19 May 2020 at 09:26, Jakub Jelinek via Gcc <gcc@gcc.gnu.org> wrote:
>

> On Tue, May 19, 2020 at 10:11:28AM +0200, Martin Liška wrote:

> > > I find this format more helpful for the reasons below so unless your

> > > script can be tweaked to do something similar I'd like to be able to

> > > continue to use mine going forward with the new infrastructure.

> >

> > Let's extend the contrib script.

>

> BTW, concerning mklog, the very common problem is that it doesn't do the

> right thing because the patch doesn't contain enough context to figure out

> what exactly has changed.  If the script would be used together with git

> rather than just on a patch file, perhaps it could handle more, like

> ask git for a patch with unlimited context (like -U100000000 on patch does).

> The common problems I remember is that e.g. when changing a function comment

> above some function, it is attributed to the previous function rather than

> following, labels in function confusing it:

>  void

>  foo ()

>  {

>    ...

>  label:

>    ...

> -  ...

> +  ...

>  }

> will result in (label), GTY markers confusing it

>  struct GTY foobar {

>    ...

> -  ...

> +  ...

>  };

> resulting in (struct GTY) or so, another common problem is too large

> function names (or more often *.md define_* names); here I'm afraid

> diff doesn't have an argument to not truncate it, or sometimes e.g. changes

> to #define being attributed to something else.

> I know some of the issues can be pretty hard to deal with.


This isn't a complaint, because I don't expect them to work, but
mklog.py (and diff --show-c-function) are useless for libstdc++ code,
they can't handle namespaces and result in nonsense like:

--- a/libstdc++-v3/src/c++11/random.cc
+++ b/libstdc++-v3/src/c++11/random.cc
@@ -97,7 +97,7 @@ namespace std _GLIBCXX_VISIBILITY(default)

and

libstdc++-v3/ChangeLog:

        * src/c++11/random.cc (_GLIBCXX_VISIBILITY):
        * testsuite/26_numerics/random/random_device/94087.cc: New test.
Michael Matz May 19, 2020, 9:56 p.m. | #21
Hello,

On Tue, 19 May 2020, Jakub Jelinek wrote:

> On Tue, May 19, 2020 at 05:21:16PM +0100, Richard Earnshaw wrote:

> > This is really a wart in the GNU coding style.  And one reason why I

> > tend to indent such labels by a single space.  It particularly affects

> > things like class definitions where public, private, etc statements

> > often appear in column 0.

> > 

> > IMO, it would be nice to get an official change in the coding style for

> > this, it's really irritating.

> 

> It doesn't have to be just label,

> void

> foo ()

> {

>   ...

> #define X ...

>   ...

> #undef X

>   ...

> }

> does the similar thing for mklog.


That particular one would be a mere bug in mklog then.  diff -p regards 
only members of [[:alpha:]$_] as acceptable start characters of function 
names (i.e. indeed things that can start a C identifier (ignoring details 
like non-base characters) with the '$' extension), of which '#' is none.


Ciao,
Michael.
Martin Liška May 21, 2020, 8:16 a.m. | #22
Hello Martin.

Can you please compare the current mklog.py. Is there anything
you miss compared to your current script?

Thanks,
Martin
Thomas Rodgers via Gcc-patches May 21, 2020, 10:03 p.m. | #23
On Fri, May 15, 2020 at 11:39 AM Martin Liška <mliska@suse.cz> wrote:
>

> On 5/15/20 3:22 PM, Marek Polacek wrote:

> > On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:

> >> On 5/15/20 2:42 PM, Marek Polacek wrote:

> >>> I actually use mklog -i all the time.  But I can work around it if it

> >>> disappears.

> >>

> >> Ah, I can see a consumer.

> >> There's an updated version that supports that.

> >>

> >> For the future, will you still use the option? Wouldn't be better

> >> to put the ChangeLog content directly to commit message? Note

> >> that you won't have to copy the entries to a particular ChangeLog file.

> >

> > The way I do it is to generate a patch using format-patch, use mklog -i

> > on it, then add the ChangeLog entry to the commit message via commit --amend.

>

> Hmm, you can do much better with:

>

> $ git diff | ./contrib/mklog > changelog && git commit -a -t changelog

>

> Or for an already created commit you can do:

>

> $ git diff HEAD~ | ./contrib/mklog > changelog && git commit -a --amend -e -F changelog


With these git aliases:

        mklog-editor = "!f() { git show | git gcc-mklog >> $1; }; f"
        addlog = "!f() { GIT_EDITOR='git mklog-editor' git commit --amend; }; f"

I can 'git addlog' to append the output of mklog to the current
commit.  Probably better would be to do something with
prepare-commit-msg.

Jason
Thomas Rodgers via Gcc-patches May 22, 2020, 4:43 p.m. | #24
On 5/21/20 2:16 AM, Martin Liška wrote:
> Hello Martin.

> 

> Can you please compare the current mklog.py. Is there anything

> you miss compared to your current script?


Nope, it matches the format I get with my script and even works
better and runs faster.  Very nice!  I'll be happy to switch to
using it instead.

Thanks!
Martin

PS A couple of ideas for future enhancements are to have the script
print "New function." or "New type." for newly added functions and
types, and to print "Adjust comments." for changes to comments alone.
Thomas Rodgers via Gcc-patches May 22, 2020, 5:28 p.m. | #25
Hi,

what's currently in trunk (as of a few hours ago) fails for me with

   File "contrib/mklog.py", line 36, in <module>
     from unidiff import PatchSet
ModuleNotFoundError: No module named 'unidiff'

I think this is an error which would have to be taken into account
one way or another - maybe include it in download_prerequisites ?

Regards

	Thomas
Thomas Rodgers via Gcc-patches May 22, 2020, 6:48 p.m. | #26
On Fri, 22 May 2020 at 19:15, Thomas Koenig via Gcc <gcc@gcc.gnu.org> wrote:
>

> Hi,

>

> what's currently in trunk (as of a few hours ago) fails for me with

>

>    File "contrib/mklog.py", line 36, in <module>

>      from unidiff import PatchSet

> ModuleNotFoundError: No module named 'unidiff'

>

> I think this is an error which would have to be taken into account

> one way or another - maybe include it in download_prerequisites ?


No, most people do not want download_prerequisites to start installing
python packages, they should use their system's package manager.


>

> Regards

>

>         Thomas
Thomas Rodgers via Gcc-patches May 22, 2020, 9:01 p.m. | #27
On Thu, May 21, 2020 at 6:03 PM Jason Merrill <jason@redhat.com> wrote:
>

> On Fri, May 15, 2020 at 11:39 AM Martin Liška <mliska@suse.cz> wrote:

> >

> > On 5/15/20 3:22 PM, Marek Polacek wrote:

> > > On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:

> > >> On 5/15/20 2:42 PM, Marek Polacek wrote:

> > >>> I actually use mklog -i all the time.  But I can work around it if it

> > >>> disappears.

> > >>

> > >> Ah, I can see a consumer.

> > >> There's an updated version that supports that.

> > >>

> > >> For the future, will you still use the option? Wouldn't be better

> > >> to put the ChangeLog content directly to commit message? Note

> > >> that you won't have to copy the entries to a particular ChangeLog file.

> > >

> > > The way I do it is to generate a patch using format-patch, use mklog -i

> > > on it, then add the ChangeLog entry to the commit message via commit --amend.

> >

> > Hmm, you can do much better with:

> >

> > $ git diff | ./contrib/mklog > changelog && git commit -a -t changelog

> >

> > Or for an already created commit you can do:

> >

> > $ git diff HEAD~ | ./contrib/mklog > changelog && git commit -a --amend -e -F changelog

>

> With these git aliases:

>

>         mklog-editor = "!f() { git show | git gcc-mklog >> $1; }; f"

>         addlog = "!f() { GIT_EDITOR='git mklog-editor' git commit --amend; }; f"

>

> I can 'git addlog' to append the output of mklog to the current

> commit.  Probably better would be to do something with

> prepare-commit-msg.


This is pretty rudimentary, but good enough as a start:

#!/bin/sh

#COMMIT_MSG_FILE=$1
#COMMIT_SOURCE=$2
#SHA1=$3

if ! [ -f "$1" ]; then exit 0; fi

#echo "# $0 $1 $2 $3" >> $1

if fgrep 'ChangeLog:' $1 > /dev/null 2>&1; then exit 0; fi

if [ -z "$2" ]; then
    cmd="diff --cached"
elif [ $2 == commit ]; then
    cmd="show $3"
else
    exit 0
fi

git $cmd | git gcc-mklog >> $1
Martin Liška May 25, 2020, 7:54 a.m. | #28
On 5/22/20 6:43 PM, Martin Sebor wrote:
> On 5/21/20 2:16 AM, Martin Liška wrote:

>> Hello Martin.

>>

>> Can you please compare the current mklog.py. Is there anything

>> you miss compared to your current script?

> 

> Nope, it matches the format I get with my script and even works

> better and runs faster.  Very nice!  I'll be happy to switch to

> using it instead.


Great, good to hear!

> 

> Thanks!

> Martin

> 

> PS A couple of ideas for future enhancements are to have the script

> print "New function." or "New type." for newly added functions and

> types, and to print "Adjust comments." for changes to comments alone.


Feel free to send patches for the new script ;)

Martin
Martin Liška May 25, 2020, 9:23 a.m. | #29
On 5/22/20 11:01 PM, Jason Merrill wrote:
> On Thu, May 21, 2020 at 6:03 PM Jason Merrill <jason@redhat.com> wrote:

>>

>> On Fri, May 15, 2020 at 11:39 AM Martin Liška <mliska@suse.cz> wrote:

>>>

>>> On 5/15/20 3:22 PM, Marek Polacek wrote:

>>>> On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:

>>>>> On 5/15/20 2:42 PM, Marek Polacek wrote:

>>>>>> I actually use mklog -i all the time.  But I can work around it if it

>>>>>> disappears.

>>>>>

>>>>> Ah, I can see a consumer.

>>>>> There's an updated version that supports that.

>>>>>

>>>>> For the future, will you still use the option? Wouldn't be better

>>>>> to put the ChangeLog content directly to commit message? Note

>>>>> that you won't have to copy the entries to a particular ChangeLog file.

>>>>

>>>> The way I do it is to generate a patch using format-patch, use mklog -i

>>>> on it, then add the ChangeLog entry to the commit message via commit --amend.

>>>

>>> Hmm, you can do much better with:

>>>

>>> $ git diff | ./contrib/mklog > changelog && git commit -a -t changelog

>>>

>>> Or for an already created commit you can do:

>>>

>>> $ git diff HEAD~ | ./contrib/mklog > changelog && git commit -a --amend -e -F changelog

>>

>> With these git aliases:

>>

>>          mklog-editor = "!f() { git show | git gcc-mklog >> $1; }; f"

>>          addlog = "!f() { GIT_EDITOR='git mklog-editor' git commit --amend; }; f"

>>

>> I can 'git addlog' to append the output of mklog to the current

>> commit.  Probably better would be to do something with

>> prepare-commit-msg.

> 

> This is pretty rudimentary, but good enough as a start:


I like the idea of usage of the prepare commit hook.

> 

> #!/bin/sh

> 

> #COMMIT_MSG_FILE=$1

> #COMMIT_SOURCE=$2

> #SHA1=$3


It's better to use the named arguments.

> 

> if ! [ -f "$1" ]; then exit 0; fi

> 

> #echo "# $0 $1 $2 $3" >> $1

> 

> if fgrep 'ChangeLog:' $1 > /dev/null 2>&1; then exit 0; fi

> 

> if [ -z "$2" ]; then

>      cmd="diff --cached"

> elif [ $2 == commit ]; then

>      cmd="show $3"

> else

>      exit 0

> fi

> 

> git $cmd | git gcc-mklog >> $1

> 


Well, that will generate changelog entry for each commit.
For a user branch development, it's not desirable.

What about more explicit approach:

1) making an alias for: git diff | git gcc-mklog > commit.msg
2) hook:

if test -f commit.msg; then
   cat commit.msg >> "$COMMIT_MSG_FILE"
   rm commit.msg
fi

So the changelog is created explicitly and included implicitly.

Martin
Thomas Rodgers via Gcc-patches May 25, 2020, 7:41 p.m. | #30
On Mon, May 25, 2020 at 5:23 AM Martin Liška <mliska@suse.cz> wrote:
>

> On 5/22/20 11:01 PM, Jason Merrill wrote:

> > On Thu, May 21, 2020 at 6:03 PM Jason Merrill <jason@redhat.com> wrote:

> >>

> >> On Fri, May 15, 2020 at 11:39 AM Martin Liška <mliska@suse.cz> wrote:

> >>>

> >>> On 5/15/20 3:22 PM, Marek Polacek wrote:

> >>>> On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:

> >>>>> On 5/15/20 2:42 PM, Marek Polacek wrote:

> >>>>>> I actually use mklog -i all the time.  But I can work around it if it

> >>>>>> disappears.

> >>>>>

> >>>>> Ah, I can see a consumer.

> >>>>> There's an updated version that supports that.

> >>>>>

> >>>>> For the future, will you still use the option? Wouldn't be better

> >>>>> to put the ChangeLog content directly to commit message? Note

> >>>>> that you won't have to copy the entries to a particular ChangeLog file.

> >>>>

> >>>> The way I do it is to generate a patch using format-patch, use mklog -i

> >>>> on it, then add the ChangeLog entry to the commit message via commit --amend.

> >>>

> >>> Hmm, you can do much better with:

> >>>

> >>> $ git diff | ./contrib/mklog > changelog && git commit -a -t changelog

> >>>

> >>> Or for an already created commit you can do:

> >>>

> >>> $ git diff HEAD~ | ./contrib/mklog > changelog && git commit -a --amend -e -F changelog

> >>

> >> With these git aliases:

> >>

> >>          mklog-editor = "!f() { git show | git gcc-mklog >> $1; }; f"

> >>          addlog = "!f() { GIT_EDITOR='git mklog-editor' git commit --amend; }; f"

> >>

> >> I can 'git addlog' to append the output of mklog to the current

> >> commit.  Probably better would be to do something with

> >> prepare-commit-msg.

> >

> > This is pretty rudimentary, but good enough as a start:

>

> I like the idea of usage of the prepare commit hook.

>

> >

> > #!/bin/sh

> >

> > #COMMIT_MSG_FILE=$1

> > #COMMIT_SOURCE=$2

> > #SHA1=$3

>

> It's better to use the named arguments.

>

> >

> > if ! [ -f "$1" ]; then exit 0; fi

> >

> > #echo "# $0 $1 $2 $3" >> $1

> >

> > if fgrep 'ChangeLog:' $1 > /dev/null 2>&1; then exit 0; fi

> >

> > if [ -z "$2" ]; then

> >      cmd="diff --cached"

> > elif [ $2 == commit ]; then

> >      cmd="show $3"

> > else

> >      exit 0

> > fi

> >

> > git $cmd | git gcc-mklog >> $1

> >

>

> Well, that will generate changelog entry for each commit.

> For a user branch development, it's not desirable.


It isn't that useful for intermediate commits, but I was thinking it
wasn't harmful either.  But I can remove the on-by-default aspects.

> What about more explicit approach:

>

> 1) making an alias for: git diff | git gcc-mklog > commit.msg

> 2) hook:

>

> if test -f commit.msg; then

>    cat commit.msg >> "$COMMIT_MSG_FILE"

>    rm commit.msg

> fi

>

> So the changelog is created explicitly and included implicitly.


How about this?
commit 787893dc41fb8288994a2350943c22e2388476e7
Author: Jason Merrill <jason@redhat.com>
Date:   Fri May 22 18:40:35 2020 -0400

    gcc-git: Add prepare-commit-msg.
    
    This patch introduces a prepare-commit-msg hook that appends a ChangeLog
    skeleton to a commit message when the GCC_FORCE_MKLOG environment variable
    is set, and a 'git commit-mklog' command set that variable while running
    'git commit'.
    
    contrib/ChangeLog:
    
            * prepare-commit-msg: New file.
            * gcc-git-customization.sh: Install it.  Add commit-mklog alias.

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index 7a950ae5f38..a3f7da8d20b 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -30,6 +30,11 @@ git config alias.gcc-backport '!f() { rev=$1; git cherry-pick -x $@; } ; f'
 
 git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog.py" $@; } ; f'
 
+hookdir=`git rev-parse --git-path hooks`
+install "`git rev-parse --show-toplevel`/contrib/prepare-commit-msg" "$hookdir"
+
+git config alias.commit-mklog '!f() { GCC_FORCE_MKLOG=1 git commit "$@"; }; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
diff --git a/contrib/prepare-commit-msg b/contrib/prepare-commit-msg
new file mode 100644
index 00000000000..b06080b927c
--- /dev/null
+++ b/contrib/prepare-commit-msg
@@ -0,0 +1,43 @@
+#!/bin/sh
+
+COMMIT_MSG_FILE=$1
+COMMIT_SOURCE=$2
+SHA1=$3
+
+#echo "# $*" > $HOME/prepare-commit-msg-args
+
+# Can't do anything if $COMMIT_MSG_FILE isn't a file.
+if ! [ -f "$COMMIT_MSG_FILE" ]; then exit 0; fi
+
+# Don't do anything unless requested to.
+if [ -z "$GCC_FORCE_MKLOG" ]; then exit 0; fi
+
+if [ -z "$COMMIT_SOURCE" ] || [ $COMMIT_SOURCE = template ]; then
+    # No source or "template" means new commit.
+    cmd="diff --cached"
+
+elif [ $COMMIT_SOURCE = message ]; then
+    # "message" means -m, assume a new commit if there are any changes staged.
+    if ! git diff --cached --quiet; then
+	cmd="diff --cached"
+    else
+	cmd="diff --cached HEAD^"
+    fi
+
+    # Add a blank line before the ChangeLog entries.
+    echo >> "$COMMIT_MSG_FILE"
+
+elif [ $COMMIT_SOURCE = commit ]; then
+    # The message of an existing commit.  If it's HEAD, assume --amend;
+    # otherwise, assume a new commit with -C.
+    if [ $SHA1 = HEAD ]; then
+	cmd="diff --cached HEAD^"
+    else
+	cmd="diff --cached"
+    fi
+else
+    # Do nothing for merge or squash.
+    exit 0
+fi
+
+git $cmd | git gcc-mklog >> "$COMMIT_MSG_FILE"
Richard Earnshaw May 26, 2020, 10:23 a.m. | #31
On 25/05/2020 20:41, Jason Merrill via Gcc-patches wrote:
> On Mon, May 25, 2020 at 5:23 AM Martin Liška <mliska@suse.cz> wrote:

>>

>> On 5/22/20 11:01 PM, Jason Merrill wrote:

>>> On Thu, May 21, 2020 at 6:03 PM Jason Merrill <jason@redhat.com> wrote:

>>>>

>>>> On Fri, May 15, 2020 at 11:39 AM Martin Liška <mliska@suse.cz> wrote:

>>>>>

>>>>> On 5/15/20 3:22 PM, Marek Polacek wrote:

>>>>>> On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:

>>>>>>> On 5/15/20 2:42 PM, Marek Polacek wrote:

>>>>>>>> I actually use mklog -i all the time.  But I can work around it if it

>>>>>>>> disappears.

>>>>>>>

>>>>>>> Ah, I can see a consumer.

>>>>>>> There's an updated version that supports that.

>>>>>>>

>>>>>>> For the future, will you still use the option? Wouldn't be better

>>>>>>> to put the ChangeLog content directly to commit message? Note

>>>>>>> that you won't have to copy the entries to a particular ChangeLog file.

>>>>>>

>>>>>> The way I do it is to generate a patch using format-patch, use mklog -i

>>>>>> on it, then add the ChangeLog entry to the commit message via commit --amend.

>>>>>

>>>>> Hmm, you can do much better with:

>>>>>

>>>>> $ git diff | ./contrib/mklog > changelog && git commit -a -t changelog

>>>>>

>>>>> Or for an already created commit you can do:

>>>>>

>>>>> $ git diff HEAD~ | ./contrib/mklog > changelog && git commit -a --amend -e -F changelog

>>>>

>>>> With these git aliases:

>>>>

>>>>          mklog-editor = "!f() { git show | git gcc-mklog >> $1; }; f"

>>>>          addlog = "!f() { GIT_EDITOR='git mklog-editor' git commit --amend; }; f"

>>>>

>>>> I can 'git addlog' to append the output of mklog to the current

>>>> commit.  Probably better would be to do something with

>>>> prepare-commit-msg.

>>>

>>> This is pretty rudimentary, but good enough as a start:

>>

>> I like the idea of usage of the prepare commit hook.

>>

>>>

>>> #!/bin/sh

>>>

>>> #COMMIT_MSG_FILE=$1

>>> #COMMIT_SOURCE=$2

>>> #SHA1=$3

>>

>> It's better to use the named arguments.

>>

>>>

>>> if ! [ -f "$1" ]; then exit 0; fi

>>>

>>> #echo "# $0 $1 $2 $3" >> $1

>>>

>>> if fgrep 'ChangeLog:' $1 > /dev/null 2>&1; then exit 0; fi

>>>

>>> if [ -z "$2" ]; then

>>>      cmd="diff --cached"

>>> elif [ $2 == commit ]; then

>>>      cmd="show $3"

>>> else

>>>      exit 0

>>> fi

>>>

>>> git $cmd | git gcc-mklog >> $1

>>>

>>

>> Well, that will generate changelog entry for each commit.

>> For a user branch development, it's not desirable.

> 

> It isn't that useful for intermediate commits, but I was thinking it

> wasn't harmful either.  But I can remove the on-by-default aspects.

> 

>> What about more explicit approach:

>>

>> 1) making an alias for: git diff | git gcc-mklog > commit.msg

>> 2) hook:

>>

>> if test -f commit.msg; then

>>    cat commit.msg >> "$COMMIT_MSG_FILE"

>>    rm commit.msg

>> fi

>>

>> So the changelog is created explicitly and included implicitly.

> 

> How about this?

> 


> +git config alias.commit-mklog '!f() { GCC_FORCE_MKLOG=1 git commit

"$@"; }; f'
> +


I thought we had a convention that aliases we added were prefixed with
'gcc-'?  This seems to go against that.

R.
Martin Liška May 26, 2020, 11:14 a.m. | #32
On 5/26/20 12:23 PM, Richard Earnshaw wrote:
> I thought we had a convention that aliases we added were prefixed with

> 'gcc-'?  This seems to go against that.


You are right, but this one is so handy ;)
What name do you suggest?

Martin
Richard Earnshaw May 26, 2020, 11:18 a.m. | #33
On 26/05/2020 12:14, Martin Liška wrote:
> On 5/26/20 12:23 PM, Richard Earnshaw wrote:

>> I thought we had a convention that aliases we added were prefixed with

>> 'gcc-'?  This seems to go against that.

> 

> You are right, but this one is so handy ;)

> What name do you suggest?

> 

> Martin


gcc-ci?

R.
Martin Liška May 26, 2020, 1:09 p.m. | #34
On 5/26/20 1:18 PM, Richard Earnshaw wrote:
> On 26/05/2020 12:14, Martin Liška wrote:

>> On 5/26/20 12:23 PM, Richard Earnshaw wrote:

>>> I thought we had a convention that aliases we added were prefixed with

>>> 'gcc-'?  This seems to go against that.

>>

>> You are right, but this one is so handy ;)

>> What name do you suggest?

>>

>> Martin

> 

> gcc-ci?


What the abbreviation stands for?

Martin

> 

> R.

>
Richard Earnshaw May 26, 2020, 1:11 p.m. | #35
On 26/05/2020 14:09, Martin Liška wrote:
> On 5/26/20 1:18 PM, Richard Earnshaw wrote:

>> On 26/05/2020 12:14, Martin Liška wrote:

>>> On 5/26/20 12:23 PM, Richard Earnshaw wrote:

>>>> I thought we had a convention that aliases we added were prefixed with

>>>> 'gcc-'?  This seems to go against that.

>>>

>>> You are right, but this one is so handy ;)

>>> What name do you suggest?

>>>

>>> Martin

>>

>> gcc-ci?

> 

> What the abbreviation stands for?

> 

> Martin

> 

>>

>> R.

>>

> 


CheckIn

For those who come from the SVN days where ci was the standard
abbreviation for committing :-)

R.
Martin Liška May 26, 2020, 1:14 p.m. | #36
On 5/26/20 3:11 PM, Richard Earnshaw wrote:
> On 26/05/2020 14:09, Martin Liška wrote:

>> On 5/26/20 1:18 PM, Richard Earnshaw wrote:

>>> On 26/05/2020 12:14, Martin Liška wrote:

>>>> On 5/26/20 12:23 PM, Richard Earnshaw wrote:

>>>>> I thought we had a convention that aliases we added were prefixed with

>>>>> 'gcc-'?  This seems to go against that.

>>>>

>>>> You are right, but this one is so handy ;)

>>>> What name do you suggest?

>>>>

>>>> Martin

>>>

>>> gcc-ci?

>>

>> What the abbreviation stands for?

>>

>> Martin

>>

>>>

>>> R.

>>>

>>

> 

> CheckIn

> 

> For those who come from the SVN days where ci was the standard

> abbreviation for committing :-)


Ah, I see. Anyway, I prefer the original name even though it violates
the naming policy.

Let other express their preferences (Jason?).

Martin

> 

> R.

>
Thomas Rodgers via Gcc-patches May 26, 2020, 1:18 p.m. | #37
On Tue, May 26, 2020 at 03:14:54PM +0200, Martin Liška wrote:
> > > > gcc-ci?

> > > 

> > > What the abbreviation stands for?

> > > 

> > > Martin

> > > 

> > > > 

> > > > R.

> > > > 

> > > 

> > 

> > CheckIn

> > 

> > For those who come from the SVN days where ci was the standard

> > abbreviation for committing :-)

> 

> Ah, I see. Anyway, I prefer the original name even though it violates

> the naming policy.

> 

> Let other express their preferences (Jason?).


gcc-ci looks nice to me.

	Jakub
Thomas Rodgers via Gcc-patches May 26, 2020, 3:38 p.m. | #38
On 5/26/20 7:14 AM, Martin Liška wrote:
> On 5/26/20 3:11 PM, Richard Earnshaw wrote:

>> On 26/05/2020 14:09, Martin Liška wrote:

>>> On 5/26/20 1:18 PM, Richard Earnshaw wrote:

>>>> On 26/05/2020 12:14, Martin Liška wrote:

>>>>> On 5/26/20 12:23 PM, Richard Earnshaw wrote:

>>>>>> I thought we had a convention that aliases we added were prefixed 

>>>>>> with

>>>>>> 'gcc-'?  This seems to go against that.

>>>>>

>>>>> You are right, but this one is so handy ;)

>>>>> What name do you suggest?

>>>>>

>>>>> Martin

>>>>

>>>> gcc-ci?

>>>

>>> What the abbreviation stands for?

>>>

>>> Martin

>>>

>>>>

>>>> R.

>>>>

>>>

>>

>> CheckIn

>>

>> For those who come from the SVN days where ci was the standard

>> abbreviation for committing :-)

> 

> Ah, I see. Anyway, I prefer the original name even though it violates

> the naming policy.

> 

> Let other express their preferences (Jason?).


Personally I struggle to keep track of all the different command
and function names (e.g., in GCC) and what they do.  I find
consistent names and behaviors helpful (and conversely,
inconsistencies trip me up time and time again).  So if/since there
is a convention to prefix gcc Git commands with gcc- I would prefer
to use it consistently, even if shorter names might seem nicer or
more convenient.  We don't check things in so often that typing
a few extra characters should be a burden.

By the way, it's nice that the existing gcc- aliases are documented
on https://gcc.gnu.org/gitwrite.html.  I would suggest to add this
one there as well.

Martin
Thomas Rodgers via Gcc-patches May 26, 2020, 6:06 p.m. | #39
On Tue, May 26, 2020 at 11:38 AM Martin Sebor <msebor@gmail.com> wrote:

> On 5/26/20 7:14 AM, Martin Liška wrote:

> > On 5/26/20 3:11 PM, Richard Earnshaw wrote:

> >> On 26/05/2020 14:09, Martin Liška wrote:

> >>> On 5/26/20 1:18 PM, Richard Earnshaw wrote:

> >>>> On 26/05/2020 12:14, Martin Liška wrote:

> >>>>> On 5/26/20 12:23 PM, Richard Earnshaw wrote:

> >>>>>> I thought we had a convention that aliases we added were prefixed

> >>>>>> with 'gcc-'?  This seems to go against that.

> >>>>>

> >>>>> You are right, but this one is so handy ;)

> >>>>> What name do you suggest?

> >>>>

> >>>> gcc-ci?

> >>>

> >>> What the abbreviation stands for?

> >>

> >> CheckIn

> >>

> >> For those who come from the SVN days where ci was the standard

> >> abbreviation for committing :-)

> >

> > Ah, I see. Anyway, I prefer the original name even though it violates

> > the naming policy.

> >

> > Let other express their preferences (Jason?).

>


gcc-ci suggests that it should always be used for commits to gcc, which is
not my intent; the new alias is only used when you want to ask for a new
ChangeLog skeleton to be added.

gcc-ci-log would be better if you want something short; I'd prefer
gcc-commit-mklog, and let people define their own shorter aliases as
desired.


> Personally I struggle to keep track of all the different command

> and function names (e.g., in GCC) and what they do.  I find

> consistent names and behaviors helpful (and conversely,

> inconsistencies trip me up time and time again).  So if/since there

> is a convention to prefix gcc Git commands with gcc- I would prefer

> to use it consistently, even if shorter names might seem nicer or

> more convenient.  We don't check things in so often that typing

> a few extra characters should be a burden.

>


And people can always create their own aliases.


> By the way, it's nice that the existing gcc- aliases are documented

> on https://gcc.gnu.org/gitwrite.html.  I would suggest to add this

> one there as well.

>


Definitely.

Jason
Martin Liška May 27, 2020, 8:17 a.m. | #40
On 5/26/20 8:06 PM, Jason Merrill wrote:
> gcc-ci-log would be better if you want something short; I'd prefer gcc-commit-mklog, and let people define their own shorter aliases as desired.


Hello.

There's a rename I'm going to install.

Martin
From b423f910dcc2a58a86b61cc5b966a81066abbf12 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>

Date: Wed, 27 May 2020 10:16:21 +0200
Subject: [PATCH] Rename commit-mklog alias to gcc-commit-mklog.

contrib/ChangeLog:

	* gcc-git-customization.sh: Rename
	commit-mklog to gcc-commit-mklog.
---
 contrib/gcc-git-customization.sh | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index dcc42683fa6..0e56dcf9873 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -27,10 +27,8 @@ git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream)
 
 git config alias.gcc-verify '!f() { "`git rev-parse --show-toplevel`/contrib/gcc-changelog/git_check_commit.py" $@; } ; f'
 git config alias.gcc-backport '!f() { rev=$1; git cherry-pick -x $@; } ; f'
-
 git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog.py" $@; } ; f'
-
-git config alias.commit-mklog '!f() { GCC_FORCE_MKLOG=1 git commit "$@"; }; f'
+git config alias.gcc-commit-mklog '!f() { GCC_FORCE_MKLOG=1 git commit "$@"; }; f'
 
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
-- 
2.26.2
Martin Liška May 27, 2020, 8:23 a.m. | #41
On 5/26/20 5:38 PM, Martin Sebor wrote:
> By the way, it's nice that the existing gcc- aliases are documented

> on https://gcc.gnu.org/gitwrite.html.  I would suggest to add this

> one there as well.


Yes. I added the documentation bit and pushed ho master.

Martin
From 035bdc56110914329c860870e338463793fb5597 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>

Date: Wed, 27 May 2020 10:22:01 +0200
Subject: [PATCH] Document gcc-commit-mklog hook.

---
 htdocs/gitwrite.html | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/htdocs/gitwrite.html b/htdocs/gitwrite.html
index e3a1305b..b9bcb768 100644
--- a/htdocs/gitwrite.html
+++ b/htdocs/gitwrite.html
@@ -436,6 +436,8 @@ repository:</p>
   <li><i>gcc-verify</i> - verify ChangeLog format for a particular commit</li>
   <li><i>gcc-backport</i> - alias for <code>git cherry-pick -x</code></li>
   <li><i>gcc-mklog</i> - generate a ChangeLog template for a patch</li>
+  <li><i>gcc-commit-mklog</i> - commit a git revision with a pre-filled
+      ChangeLog template</li>
 </ul>
 
 <p>The final customization that the script makes is to add a diff rule so
-- 
2.26.2

Patch

From 9fa5d13856f0f5ba153801baf57d4a732829f609 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Fri, 15 May 2020 00:44:07 +0200
Subject: [PATCH] Add mklog-ng.py and gcc-mklog git alias.

contrib/ChangeLog:

	* gcc-git-customization.sh: Add gcc-mklog alias.
	* mklog_ng.py: New file.
	* test_mklog_ng.py: New file.
---
 contrib/gcc-git-customization.sh |   2 +
 contrib/mklog_ng.py              | 192 +++++++++++++++++++++++++++++++
 contrib/test_mklog_ng.py         | 158 +++++++++++++++++++++++++
 3 files changed, 352 insertions(+)
 create mode 100755 contrib/mklog_ng.py
 create mode 100755 contrib/test_mklog_ng.py

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index a932bf8c06a..b7b97327be3 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -25,6 +25,8 @@  git config alias.svn-rev '!f() { rev=$1; shift; git log --all --grep="^From-SVN:
 git config alias.gcc-descr \!"f() { if test \${1:-no} = --full; then c=\${2:-master}; r=\$(git describe --all --abbrev=40 --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-,r,p'); expr match \${r:-no} '^r[0-9]\\+\$' >/dev/null && r=\${r}-0-g\$(git rev-parse \${2:-master}); else c=\${1:-master}; r=\$(git describe --all --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)-\\([0-9]\\+\\)-g[0-9a-f]*\$,r\\2-\\3,p;s,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)\$,r\\2-0,p'); fi; if test -n \$r; then o=\$(git config --get gcc-config.upstream); rr=\$(echo \$r | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\\(-g[0-9a-f]\\+\\)\\?\$,\\1,p'); if git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$rr >/dev/null; then m=releases/gcc-\$rr; else m=master; fi; git merge-base --is-ancestor \$c \${o:-origin}/\$m && \echo \${r}; fi; }; f"
 git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream); r=\$(echo \$1 | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\$,\\1,p'); n=\$(echo \$1 | sed -n 's,^r[0-9]\\+-\\([0-9]\\+\\)\$,\\1,p'); test -z \$r && echo Invalid id \$1 && exit 1; h=\$(git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$r); test -z \$h && h=\$(git rev-parse --verify --quiet \${o:-origin}/master); p=\$(git describe --all --match 'basepoints/gcc-'\$r \$h | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+-\\([0-9]\\+\\)-g[0-9a-f]*\$,\\2,p;s,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+\$,0,p'); git rev-parse --verify \$h~\$(expr \$p - \$n); }; f"
 
+git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog_ng.py" $@; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
diff --git a/contrib/mklog_ng.py b/contrib/mklog_ng.py
new file mode 100755
index 00000000000..a67fc007759
--- /dev/null
+++ b/contrib/mklog_ng.py
@@ -0,0 +1,192 @@ 
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import argparse
+import os
+import re
+import sys
+
+from unidiff import PatchSet
+
+pr_regex = re.compile(r'(\/(\/|\*)|[Cc*!])\s+(?P<pr>PR [a-z+-]+\/[0-9]+)')
+identifier_regex = re.compile(r'^([a-zA-Z0-9_#].*)')
+comment_regex = re.compile(r'^\/\*')
+struct_regex = re.compile(r'^((class|struct|union|enum)\s+[a-zA-Z0-9_]+)')
+macro_regex = re.compile(r'#\s*(define|undef)\s+([a-zA-Z0-9_]+)')
+super_macro_regex = re.compile(r'^DEF[A-Z0-9_]+\s*\(([a-zA-Z0-9_]+)')
+fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
+template_and_param_regex = re.compile(r'<[^<>]*>')
+
+function_extensions = set(['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def'])
+
+help_message = """\
+Generate ChangeLog template for PATCH.
+PATCH must be generated using diff(1)'s -up or -cp options
+(or their equivalent in git).
+"""
+
+script_folder = os.path.realpath(__file__)
+gcc_root = os.path.dirname(os.path.dirname(script_folder))
+
+
+def find_changelog(path):
+    folder = os.path.split(path)[0]
+    while True:
+        if os.path.exists(os.path.join(gcc_root, folder, 'ChangeLog')):
+            return folder
+        folder = os.path.dirname(folder)
+        if folder == '':
+            return folder
+    raise AssertionError()
+
+
+def extract_function_name(line):
+    if comment_regex.match(line):
+        return None
+    m = struct_regex.search(line)
+    if m:
+        # Struct declaration
+        return m.group(1)
+    m = macro_regex.search(line)
+    if m:
+        # Macro definition
+        return m.group(2)
+    m = super_macro_regex.search(line)
+    if m:
+        # Supermacro
+        return m.group(1)
+    m = fn_regex.search(line)
+    if m:
+        # Discard template and function parameters.
+        fn = m.group(1)
+        fn = re.sub(template_and_param_regex, '', fn)
+        return fn.rstrip()
+    return None
+
+
+def try_add_function(functions, line):
+    fn = extract_function_name(line)
+    if fn and fn not in functions:
+        functions.append(fn)
+    return bool(fn)
+
+
+def generate_changelog(data, no_functions=False):
+    changelogs = {}
+    sorted_changelogs = []
+    prs = []
+    out = ''
+    diff = PatchSet(data)
+
+    for file in diff:
+        changelog = find_changelog(file.path)
+        if changelog not in changelogs:
+            changelogs[changelog] = []
+            sorted_changelogs.append(changelog)
+        changelogs[changelog].append(file)
+
+        if 'testsuite' in file.path and file.is_added_file:
+            for line in list(file)[0]:
+                m = pr_regex.search(line.value)
+                if m:
+                    pr = m.group('pr')
+                    if pr not in prs:
+                        prs.append(pr)
+                else:
+                    break
+
+    for changelog in sorted_changelogs:
+        files = changelogs[changelog]
+        out += '%s:\n' % os.path.join(changelog, 'ChangeLog')
+        out += '\n'
+        for pr in prs:
+            out += '\t%s\n' % pr
+        for file in files:
+            assert file.path.startswith(changelog)
+            in_tests = 'testsuite' in changelog or 'testsuite' in file.path
+            relative_path = file.path[len(changelog):].lstrip('/')
+            functions = []
+            if file.is_added_file:
+                msg = 'New test' if in_tests else 'New file'
+                out += '\t* %s: %s.\n' % (relative_path, msg)
+            elif file.is_removed_file:
+                out += '\t* %s: Removed.\n' % (relative_path)
+            else:
+                if not no_functions:
+                    for hunk in file:
+                        # Do not add function names for testsuite files
+                        extension = os.path.splitext(relative_path)[1]
+                        if not in_tests and extension in function_extensions:
+                            last_fn = None
+                            modified_visited = False
+                            success = False
+                            for line in hunk:
+                                m = identifier_regex.match(line.value)
+                                if line.is_added or line.is_removed:
+                                    if not line.value.strip():
+                                        continue
+                                    modified_visited = True
+                                    if m and try_add_function(functions,
+                                                              m.group(1)):
+                                        last_fn = None
+                                        success = True
+                                elif line.is_context:
+                                    if last_fn and modified_visited:
+                                        try_add_function(functions, last_fn)
+                                        last_fn = None
+                                        modified_visited = False
+                                        success = True
+                                    elif m:
+                                        last_fn = m.group(1)
+                                        modified_visited = False
+                            if not success:
+                                try_add_function(functions,
+                                                 hunk.section_header)
+                if functions:
+                    out += '\t* %s (%s):\n' % (relative_path, functions[0])
+                    for fn in functions[1:]:
+                        out += '\t(%s):\n' % fn
+                else:
+                    out += '\t* %s:\n' % relative_path
+        out += '\n'
+    return out
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=help_message)
+    parser.add_argument('input', nargs='?',
+                        help='Patch file (or missing, read standard input)')
+    parser.add_argument('-s', '--no-functions', action='store_true',
+                        help='Do not generate function names in ChangeLogs')
+    args = parser.parse_args()
+    if args.input == '-':
+        args.input = None
+
+    input = open(args.input) if args.input else sys.stdin
+    output = generate_changelog(input.read(), args.no_functions)
+    print(output, end='')
diff --git a/contrib/test_mklog_ng.py b/contrib/test_mklog_ng.py
new file mode 100755
index 00000000000..6e68277b1bb
--- /dev/null
+++ b/contrib/test_mklog_ng.py
@@ -0,0 +1,158 @@ 
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import unittest
+
+from mklog_ng import generate_changelog
+
+PATCH1 = '''\
+diff --git a/gcc/config/riscv/riscv.h b/gcc/config/riscv/riscv.h
+index 567c23380fe..e6209ede9d6 100644
+--- a/gcc/config/riscv/riscv.h
++++ b/gcc/config/riscv/riscv.h
+@@ -920,6 +920,7 @@ extern unsigned riscv_stack_boundary;
+ #define SHIFT_RS1 15
+ #define SHIFT_IMM 20
+ #define IMM_BITS 12
++#define C_S_BITS 5
+ #define C_SxSP_BITS 6
+ 
+ #define IMM_REACH (1LL << IMM_BITS)
+@@ -929,6 +930,10 @@ extern unsigned riscv_stack_boundary;
+ #define SWSP_REACH (4LL << C_SxSP_BITS)
+ #define SDSP_REACH (8LL << C_SxSP_BITS)
+ 
++/* This is the maximum value that can be represented in a compressed load/store
++   offset (an unsigned 5-bit value scaled by 4).  */
++#define CSW_MAX_OFFSET ((4LL << C_S_BITS) - 1) & ~3
++
+ /* Called from RISCV_REORG, this is defined in riscv-sr.c.  */
+ 
+ extern void riscv_remove_unneeded_save_restore_calls (void);
+
+'''
+
+EXPECTED1 = '''\
+gcc/ChangeLog:
+
+	* config/riscv/riscv.h (C_S_BITS):
+	(CSW_MAX_OFFSET):
+
+'''
+
+PATCH2 = '''\
+diff --git a/gcc/targhooks.h b/gcc/targhooks.h
+index 9704d23f1db..b572a36e8cf 100644
+--- a/gcc/targhooks.h
++++ b/gcc/targhooks.h
+@@ -120,7 +120,7 @@ extern bool default_empty_mask_is_expensive (unsigned);
+ extern void *default_init_cost (class loop *);
+ extern unsigned default_add_stmt_cost (class vec_info *, void *, int,
+ 				       enum vect_cost_for_stmt,
+-				       class _stmt_vec_info *, int,
++				       class _stmt_vec_info *, tree, int,
+ 				       enum vect_cost_model_location);
+ extern void default_finish_cost (void *, unsigned *, unsigned *, unsigned *);
+ extern void default_destroy_cost_data (void *);
+@@ -186,6 +186,7 @@ extern tree default_emutls_var_init (tree, tree, tree);
+ extern unsigned int default_hard_regno_nregs (unsigned int, machine_mode);
+ extern bool default_hard_regno_scratch_ok (unsigned int);
+ extern bool default_mode_dependent_address_p (const_rtx, addr_space_t);
++extern bool default_new_address_profitable_p (rtx, rtx_insn *, rtx);
+ extern bool default_target_option_valid_attribute_p (tree, tree, tree, int);
+ extern bool default_target_option_pragma_parse (tree, tree);
+ extern bool default_target_can_inline_p (tree, tree);
+
+'''
+
+EXPECTED2 = '''\
+gcc/ChangeLog:
+
+	* targhooks.h (default_add_stmt_cost):
+	(default_new_address_profitable_p):
+
+'''
+
+PATCH3 = '''\
+diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
+index 2b1e33f94ae..7f47402f9b9 100644
+--- a/libcpp/include/cpplib.h
++++ b/libcpp/include/cpplib.h
+@@ -173,7 +173,7 @@ enum c_lang {CLK_GNUC89 = 0, CLK_GNUC99, CLK_GNUC11, CLK_GNUC17, CLK_GNUC2X,
+ 	     CLK_STDC2X,
+ 	     CLK_GNUCXX, CLK_CXX98, CLK_GNUCXX11, CLK_CXX11,
+ 	     CLK_GNUCXX14, CLK_CXX14, CLK_GNUCXX17, CLK_CXX17,
+-	     CLK_GNUCXX2A, CLK_CXX2A, CLK_ASM};
++	     CLK_GNUCXX20, CLK_CXX20, CLK_ASM};
+ 
+ /* Payload of a NUMBER, STRING, CHAR or COMMENT token.  */
+ struct GTY(()) cpp_string {
+@@ -484,7 +484,7 @@ struct cpp_options
+   /* Nonzero for C2X decimal floating-point constants.  */
+   unsigned char dfp_constants;
+ 
+-  /* Nonzero for C++2a __VA_OPT__ feature.  */
++  /* Nonzero for C++20 __VA_OPT__ feature.  */
+   unsigned char va_opt;
+ 
+   /* Nonzero for the '::' token.  */
+
+'''
+
+EXPECTED3 = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h (enum c_lang):
+	(struct cpp_options):
+
+'''
+
+EXPECTED3B = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h:
+
+'''
+
+
+class TestMklog(unittest.TestCase):
+    def test_macro_definition(self):
+        changelog = generate_changelog(PATCH1)
+        assert changelog == EXPECTED1
+
+    def test_changed_argument(self):
+        changelog = generate_changelog(PATCH2)
+        assert changelog == EXPECTED2
+
+    def test_enum_and_struct(self):
+        changelog = generate_changelog(PATCH3)
+        assert changelog == EXPECTED3
+
+    def test_no_function(self):
+        changelog = generate_changelog(PATCH3, True)
+        assert changelog == EXPECTED3B
-- 
2.26.2