java.util.regex.PatternSyntaxException for HelmIgnorePathMatcher.addPatterns()

When I use sample codes to install helm charts, I got below exception:

java.util.regex.PatternSyntaxException: Unclosed character class near index 23
^templates/\.[^\]?[^\]*$
                       ^
	at java.util.regex.Pattern.error(Pattern.java:1957)
	at java.util.regex.Pattern.clazz(Pattern.java:2550)
	at java.util.regex.Pattern.clazz(Pattern.java:2506)
	at java.util.regex.Pattern.sequence(Pattern.java:2065)
	at java.util.regex.Pattern.expr(Pattern.java:1998)
	at java.util.regex.Pattern.compile(Pattern.java:1698)
	at java.util.regex.Pattern.<init>(Pattern.java:1351)
	at java.util.regex.Pattern.compile(Pattern.java:1028)
	at org.microbean.helm.chart.HelmIgnorePathMatcher.addPatterns(HelmIgnorePathMatcher.java:274)
	at org.microbean.helm.chart.HelmIgnorePathMatcher.addPattern(HelmIgnorePathMatcher.java:189)
	at org.microbean.helm.chart.HelmIgnorePathMatcher.<init>(HelmIgnorePathMatcher.java:98)
	at org.microbean.helm.chart.HelmIgnorePathMatcher.<init>(HelmIgnorePathMatcher.java:112)
	at org.microbean.helm.chart.HelmIgnorePathMatcher.<init>(HelmIgnorePathMatcher.java:163)
	at org.microbean.helm.chart.DirectoryChartLoader$PathWalker.<init>(DirectoryChartLoader.java:186)
	at org.microbean.helm.chart.DirectoryChartLoader$PathWalker.<init>(DirectoryChartLoader.java:156)
	at org.microbean.helm.chart.DirectoryChartLoader.toNamedInputStreamEntries(DirectoryChartLoader.java:145)
	at org.microbean.helm.chart.DirectoryChartLoader.toNamedInputStreamEntries(DirectoryChartLoader.java:51)
	at org.microbean.helm.chart.StreamOrientedChartLoader.load(StreamOrientedChartLoader.java:213)
	at org.microbean.helm.chart.AbstractChartLoader.load(AbstractChartLoader.java:87)
	at com.microsoft.jenkins.devspaces.commands.DeployHelmChartCommand.execute(DeployHelmChartCommand.java:40)
	at com.microsoft.jenkins.devspaces.commands.DeployHelmChartCommand.execute(DeployHelmChartCommand.java:27)
	at com.microsoft.jenkins.azurecommons.command.CommandService.runCommand(CommandService.java:88)
	at com.microsoft.jenkins.azurecommons.command.CommandService.execute(CommandService.java:96)
	at com.microsoft.jenkins.azurecommons.command.CommandService.executeCommands(CommandService.java:75)
	at com.microsoft.jenkins.azurecommons.command.BaseCommandContext.executeCommands(BaseCommandContext.java:77)
	at com.microsoft.jenkins.devspaces.DevSpacesBuilder.perform(DevSpacesBuilder.java:91)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
	at hudson.model.Build$BuildExecution.build(Build.java:205)
	at hudson.model.Build$BuildExecution.doRun(Build.java:162)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
	at hudson.model.Run.execute(Run.java:1741)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:410)

It seems that the default constructor of HelmIgnorePathMatcher will try to call addPattern("templates/.?*") method and it will generate a regex pattern like ^templates/\.[^\]?[^\]*$ which is an invalid one. Any help here?

Something is odd here; note that this line in the relevant unit test works:

microbean-helm/src/test/java/org/microbean/helm/chart/TestHelmIgnorePathMatcher.java

Line 114 in 183f5f5

    
           final HelmIgnorePathMatcher helmIgnorePathMatcher = new HelmIgnorePathMatcher();

It is called here:

microbean-helm/src/main/java/org/microbean/helm/chart/HelmIgnorePathMatcher.java

Line 98 in 183f5f5

this.addPattern("templates/.?*");

I wonder if it is a JDK version thing? Do you have more details about your situation?

My jdk version is

openjdk version "1.8.0_181"
OpenJDK Runtime Environment (build 1.8.0_181-8u181-b13-1ubuntu0.16.04.1-b13)
OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)

If you want to test in this environment, you should specify the surefire plugin version as 3.0.0-M1, more information SUREFIRE-1588.

It's going to be a problem in your .helmignore file, or a problem that this class has reading it (perhaps still a legitimate bug). It is not the default constructor that is the problem. The error is really occurring from here:

microbean-helm/src/main/java/org/microbean/helm/chart/HelmIgnorePathMatcher.java

Line 163 in 183f5f5

    
           this(helmIgnoreFile == null ? (Collection<? extends String>)null : Files.readAllLines(helmIgnoreFile, StandardCharsets.UTF_8));

That's going to read in a .helmignore file and pass the patterns to the addPattern method. Then each .helmignore pattern is translated into a regular expression.

Reverse engineering the error message, it looks like this regex:

^templates/\.[^\]?[^\]*$

…is resulting from the following .helmignore template:

templates/.?*

which doesn't make much sense to me. Could you send me the offending .helmignore file so I may write a test to reproduce the issue?

The fact that \ shows up also tells me this is Windows, not Ubuntu.

My .helmignore file:

# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*~
# Various IDEs
.project
.idea/
*.tmproj

My windows jdk version:

java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)

BTW, where does the .helmignore should be put? My file structure here is:

chart
  --templates
      --service.yaml
      ....
  --Chart.yaml
  --values.yaml
  --.helmignore

This gets weirder and weirder.

Note that the Travis build (Linux) passes:

https://travis-ci.org/microbean/microbean-helm/jobs/432367395#L821

It is running with JDK 8 build 151:

https://travis-ci.org/microbean/microbean-helm/jobs/432367395#L441

But I note that now on my (Mac) laptop with JDK 8 build 181 the test fails:

[INFO] --- maven-surefire-plugin:2.20.1:test (default-test) @ microbean-helm ---
[INFO] Toolchain in maven-surefire-plugin: JDK[/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home]
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.microbean.helm.chart.TestHelmIgnorePathMatcher
[ERROR] Tests run: 23, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.147 s <<< FAILURE! - in org.microbean.helm.chart.TestHelmIgnorePathMatcher
[ERROR] testStuff[cargo/, cargo, true](org.microbean.helm.chart.TestHelmIgnorePathMatcher)  Time elapsed: 0.009 s  <<< FAILURE!
java.lang.AssertionError: cargo/ did not match cargo expected:<true> but was:<false>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:834)
	at org.junit.Assert.assertEquals(Assert.java:118)
	at org.microbean.helm.chart.TestHelmIgnorePathMatcher.testStuff(TestHelmIgnorePathMatcher.java:128)

Investigating further but looks like maybe a regex regression in JDK 8.

Hmm, even weirder. I can now reproduce this error at will on my laptop using JDK 8 builds 144, 162 and 181. There is definitely a bug in here somewhere.

However, in order to actually release microbean-helm to Maven Central, I had to run mvn release:perform, which runs all tests and fails if any of them fail. So given that I've released many versions of microbean-helm, and given that master is (currently) what is released, and given that I haven't touched either the test case in question or HelmIgnorePathMatcher.java, what has changed to cause this test to fail? Still investigating.

Somewhat more level-headed refinement: I can't reproduce the error actually described here. And I can make all the tests pass if I run mvn test instead of mvn test -Dtest=TestHelmIgnorePathMatcher.

So the pattern compilation is working fine on environments that use / as their file separator. It is not a JVM version issue.

At issue now is what happens when the file separator is \ (your issue) and the also disturbing issue where mvn test -Dtest=TestHelmIgnorePathMatcher fails.

I got this problem too.
But this is a little weirder, it work fine on Centos7 and get this error on Windows.
Just DirectoryChartLoader.load() have this error when I want read chart from directory, TapeArchiveChartLoader.load() work fine when I read a XXX.tgz on Windows.
As we know, I can't run k8s and helm Windows, so I just ignore this error...
Error code:

String directoryPath = "C:/Users/daiqingliang/Desktop/helm-kubeapps/my-hello-world";
DirectoryChartLoader loader = new DirectoryChartLoader();
ChartOuterClass.Chart.Builder chart = loader.load(Paths.get(directoryPath));
MetadataOuterClass.MetadataOrBuilder metadata = chart.getMetadataOrBuilder();

exception:

java.util.regex.PatternSyntaxException: Unclosed character class near index 23
^templates/\.[^\]?[^\]*$
                       ^
	at java.util.regex.Pattern.error(Pattern.java:1957)
	at java.util.regex.Pattern.clazz(Pattern.java:2550)
	at java.util.regex.Pattern.clazz(Pattern.java:2506)
	at java.util.regex.Pattern.sequence(Pattern.java:2065)
	at java.util.regex.Pattern.expr(Pattern.java:1998)
	at java.util.regex.Pattern.compile(Pattern.java:1698)
	at java.util.regex.Pattern.<init>(Pattern.java:1351)
	at java.util.regex.Pattern.compile(Pattern.java:1028)
	at org.microbean.helm.chart.HelmIgnorePathMatcher.addPatterns(HelmIgnorePathMatcher.java:274)
	at org.microbean.helm.chart.HelmIgnorePathMatcher.addPattern(HelmIgnorePathMatcher.java:189)
	at org.microbean.helm.chart.HelmIgnorePathMatcher.<init>(HelmIgnorePathMatcher.java:98)
	at org.microbean.helm.chart.HelmIgnorePathMatcher.<init>(HelmIgnorePathMatcher.java:112)
	at org.microbean.helm.chart.HelmIgnorePathMatcher.<init>(HelmIgnorePathMatcher.java:163)
	at org.microbean.helm.chart.DirectoryChartLoader$PathWalker.<init>(DirectoryChartLoader.java:186)
	at org.microbean.helm.chart.DirectoryChartLoader$PathWalker.<init>(DirectoryChartLoader.java:156)
	at org.microbean.helm.chart.DirectoryChartLoader.toNamedInputStreamEntries(DirectoryChartLoader.java:145)
	at org.microbean.helm.chart.DirectoryChartLoader.toNamedInputStreamEntries(DirectoryChartLoader.java:51)
	at org.microbean.helm.chart.StreamOrientedChartLoader.load(StreamOrientedChartLoader.java:213)
	at org.microbean.helm.chart.AbstractChartLoader.load(AbstractChartLoader.java:87)
	at com.hisense.kubeappshelm.mytest.TestReadChartByDireactory.TestLocalChart(TestReadChartByDireactory.java:40)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)

Process finished with exit code 0

Navigation Menu

Repository navigation

java.util.regex.PatternSyntaxException for HelmIgnorePathMatcher.addPatterns() #189

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

java.util.regex.PatternSyntaxException for HelmIgnorePathMatcher.addPatterns() #189

Description

Activity

ljnelson commented on Nov 20, 2018

gavinfish commented on Nov 21, 2018

ljnelson commented on Nov 21, 2018

gavinfish commented on Nov 21, 2018

ljnelson commented on Nov 24, 2018

ljnelson commented on Nov 24, 2018

ljnelson commented on Nov 24, 2018

daiqingliang commented on Mar 12, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions