HIVE-29662: Cannot execute a script with Beeline's -f <script>

[thomasrebele]

What changes were proposed in this pull request?

Update jline to 3.30.16, which includes the fix for HIVE-29662: Make Beeline's -f <script> CLI actually execute the script.

Why are the changes needed?

JLine had some bugs which broke Beeline's -f <script> feature.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Created a file script.sql containing some commands:

select 1;
use db;
select * from tab1;
select reflect("java.lang.Thread", "sleep", bigint(20000));

Built the PR and executed Beeline with ./beeline -u jdbc:hive2://localhost:10000 -n trebele --verbose=true -f /tmp/script.sql. The script was executed as expected.

...
+-------+
|  _c0  |
+-------+
| null  |
+-------+
1 row selected (20.185 seconds)
>....>....Closing: 0: jdbc:hive2://localhost:10000

Interactive mode works as well. The prompt line starts with 0: jdbc:hive2://localhost:10000>. I did not observe the problem described in HIVE-28992.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[ayushtkn]

LGTM

Good to get blessings from @abstractdog as well, before we push this

[abstractdog]

I'm assuming that the problem I reported in HIVE-28992 until 3.30.4 is fixed somewhere between 3.30.4 and 3.30.16
I'm tempted to blindly trust your manual testing @thomasrebele, however I would appreciate it if you could double-check that you can simply reproduce HIVE-28992 with 3.30.4 and confirm it's gone in 3.30.16 in your own testing setup, making us 100% sure that nothing is broken in this sense

[Aggarwal-Raghav]

Checked the before and after scenerio, -f flag is working ✅ but HIVE-28992 is still appearing i guess i.e >....INFO:
BEFORE:

AFTER:

[thomasrebele]

@abstractdog, I mentioned in this comment that I could not observe the problem described in HIVE-28992 for any of the versions that I've tested. I guess it depends on the operating system. To be sure that we're talking about the same thing: does HIVE-28992 refer to the interactive prompt?

[abstractdog]

@abstractdog, I mentioned in this comment that I could not observe the problem described in HIVE-28992 for any of the versions that I've tested. I guess it depends on the operating system. To be sure that we're talking about the same thing: does HIVE-28992 refer to the interactive prompt?

yeah, I meant the interactive prompt, I'll retry with jline 3.25.0, 3.25.1, 3.30.16 soon to see
btw, I'm not sure what @Aggarwal-Raghav means regarding "before" and "after", as I can see the empty prompt on both screenshots, which is >....

[thomasrebele]

Thank you! The script execution with thew new JLine version has some other problems as well. There are some spurious characters in the output (See household_de, c, c_first_shipto_date_sk big in the example below). It would also be nice if the output would print the query and then the corresponding result, then the next query, and so on. Now all queries are printed at the beginning and then the all the results. However, it this is still better than not executing the script at all. So I propose to create a follow-up ticket to fix that.

...
No rows affected (0.077 seconds)
>....>....>....>....>....>....>....>....>.... c_first_shipto_date_sk big>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....going to print operations logs
printed operations logs
Getting log thread is interrupted, since query is done!
No rows affected (0.037 seconds)
>....c>....>....>....going to print operations logs
printed operations logs
Getting log thread is interrupted, since query is done!
No rows affected (0.024 seconds)
>....>....>....>....going to print operations logs
printed operations logs
Getting log thread is interrupted, since query is done!
No rows affected (0.04 seconds)
>....>....>....>....>....>....>....>....>....>....>....>....>....household_de>....>....>....>....>....>....>....>....>....going to print operations logs
printed operations logs
Getting log thread is interrupted, since query is done!
No rows affected (0.026 seconds)
...

[abstractdog]

Thank you! The script execution with thew new JLine version has some other problems as well. There are some spurious characters in the output (See household_de, c, c_first_shipto_date_sk big in the example below). It would also be nice if the output would print the query and then the corresponding result, then the next query, and so on. Now all queries are printed at the beginning and then the all the results. However, it this is still better than not executing the script at all. So I propose to create a follow-up ticket to fix that.

...
No rows affected (0.077 seconds)
>....>....>....>....>....>....>....>....>.... c_first_shipto_date_sk big>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....>....going to print operations logs
printed operations logs
Getting log thread is interrupted, since query is done!
No rows affected (0.037 seconds)
>....c>....>....>....going to print operations logs
printed operations logs
Getting log thread is interrupted, since query is done!
No rows affected (0.024 seconds)
>....>....>....>....going to print operations logs
printed operations logs
Getting log thread is interrupted, since query is done!
No rows affected (0.04 seconds)
>....>....>....>....>....>....>....>....>....>....>....>....>....household_de>....>....>....>....>....>....>....>....>....going to print operations logs
printed operations logs
Getting log thread is interrupted, since query is done!
No rows affected (0.026 seconds)
...

okay, I'm fine with the follow-up
I just tested with different jline versions, first made sure there were no other jline jars on classpath, so I got this:

Exception in thread "main" java.lang.NoClassDefFoundError: org/jline/reader/Completer
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:534)
	at java.base/java.lang.Class.forName(Class.java:513)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:245)
Caused by: java.lang.ClassNotFoundException: org.jline.reader.Completer
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
	... 5 more

this is crucial, before that, beeline was started successfully when I removed jline, because it was present from other paths:

./apache/hadoop-3.4.2/share/hadoop/yarn/lib/jline-3.9.0.jar
./apache/tez/lib/jline-3.9.0.jar

so, after removing those, I found that all these versions were okay: 3.25.0, 3.25.1, 3.30.16
hence, I have no reason to block this PR

when I upgraded to Jline3, my environment was slightly different, but I cannot assume now that something has actually got broken between 3.25.0, 3.25.1, just got "changed", which was revealed in my old environment, but not now

I would be happy if someone can still reproduce and document under what circumstances the weird prompt >.... can appear

[abstractdog]

LGTM, assuming follow-ups

[thomasrebele]

I've created ticket HIVE-29814 as a follow-up.