Remove max_level param as mandatory

README.md

@@ -1,27 +1,24 @@
 # python-mbtiles2compactcache
 (credits: https://github.com/Esri/raster-tiles-compactcache)
 
-## Compact Cache V2 sample code
+## Compact Cache V2
 
 ### mbtiles2compactcache.py
 
-Convert individual .mbtile files to the [Esri Compact Cache V2](./CompactCacheV2.md) format bundles.  It only builds individual bundles, not a completely functional cache.
+Convert a single raster/vector tiles .mbtile file to the [Esri Compact Cache V2](./CompactCacheV2.md) format bundles. It only builds a completely functional cache. This script is designed to export large to huge raster dataset mbtiles. the export occurs using multiple threads reading all records sequentially.
 
-While operational, this code is only provided as an example of how a bundle file is created and updated.
-This Python script takes two arguments, the input mbtile folder and the output folder. It assumes that the input folder contains mbtile files of the form: ```\<level#>.mbtile```.
+Requirement:
+- data must be in Web Mercator (EPSG:3857) 
+- the tiles table must be a rowid table. (if the tiles table is a view, please add the rowid field to the view.)
 
 The script does not check the input tile format, and assumes that all the files under the source contain valid SQLLite databases with tiles in MBTiles format. 
-The algorithm loops over files, inserting each tile in the appropriate bundle. It keeps one bundle open in case the next tile fits in the same bundle.  In most cases this combination results in good performance.
+The algorithm loops over the records, inserting each tile in the appropriate bundle. Each thread writes its records in a bundle and then close it.
 
-The [sample_mbtiles](./sample_mbtiles) folder contains example [MBTiles](./sample_mbtiles/README.md) the form of SQLite databases for the single zoom levels for the first three level of the Federal Agency for Cartography and Geodesy - TopPlusOpen cache in Web Mercator projection.  The [sample_cache] (../sample_cache) folder contains a Compact Cache V2 cache produced from these individual mbtiles using the mbtiles2compactcache.py script. The commands used to generate the bundles for each level are:
+The [file](./file) folder contains example [MBTiles]
+The [cache] (./cache) folder contains a Compact Cache V2 cache produced as result of the mbtiles2compactcache.py script. The commands used to generate the cache is:
 
-RGB processing:
 ```console
-python sample_code\mbtiles2compactcache.py -i sample_mbtiles -o sample_cache\_alllayers
-```
-Grayscale processing:
-```console
-python sample_code\mbtiles2compactcache.py -i sample_mbtiles -o sample_cache\_alllayers -g
+python .\code\mbtiles2compactcache.py -ml 15 -s .\file\countries-raster.mbtiles -d .\cache\A3_MyCachedService\Layers\_alllayers
 ```
 
 ## Documentation and sample code for Esri Compact Cache V2 format
@@ -34,7 +31,7 @@ The Compact Cache V2 is the current file format used by ArcGIS to store raster t
 | Row 1 | Row 1 Col 0 | Row 1 Col 1 |
 
 ## Content
-This repository contains [documentation](CompactCacheV2.md), a [sample cache](sample_cache) and a Python 2.x [code example](sample_code) of how to build Compact Cache V2 bundles from MBTiles.
+This repository contains [documentation](CompactCacheV2.md), a [cache](cache) and a Python 3.x [code example](code) of how to build Compact Cache V2 bundles from MBTiles.
 
 ## Licensing
 

code/check_contiguous_tiles.py

@@ -0,0 +1,101 @@
+# -------------------------------------------------------------------------------
+# Name:        check_contiguous_tiles
+# Purpose:     Check if there is no hole (missing tiles) in a bundle.
+#
+# Author:      ltbam
+#
+# Created:     18/07/2022
+# Modified:    -
+#
+#  Copyright 2023 swisstopo.
+#
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.?
+#
+# -------------------------------------------------------------------------------
+# Changeset
+# Version 1.0.0 ltbam
+from mbtilesRaster2compactcache import Bundle
+import os
+import math
+
+
+cache_output_folder = r"C:\Temp\osm\A3_MyCachedService\Layers"
+
+def main():
+    print("Checking contiguous tiles in Bundle")
+    for path, subdirs, files in os.walk(cache_output_folder):
+        for name in files:
+            if name.endswith(".bundle"):
+                bdl = Bundle(os.path.join(path, name))
+                bdl.open()
+                results = listMissingTiles(bdl)
+                if len(results) == 0:
+                    print("bundle {} is ok".format(name))
+                else:
+                    for res in results:
+                        print("Missing contiguous tile: level {}, row {}, col {}".format(res["lvl"], res["row"],
+                                                                                         res["col"]))
+                # close bundle without writing anything
+                bdl.fd.close()
+                bdl.fd = None
+
+def listMissingTiles(bundle):
+    files = []
+    # Loop each Tile index and resolve if it has data
+    # range(0, 128) means 0-127
+    for row in range(0, 128):
+        startTile = 0
+        numTiles = 0
+        # count tiles with data, determine drawing center
+        for col in range(0, 128):
+            t_idx = bundle.curr_index[128 * row + col]
+            t_size = int(math.floor(t_idx / Bundle.M))
+            if t_size > 0:
+                numTiles += 1
+                if startTile == 0:
+                    startTile = col
+
+        if numTiles > 3:
+            data_started = False
+            mid_range = startTile + numTiles // 2
+            # print("lvl {} row {} mid_range: {}".format(bundle.level, row, mid_range))
+            # inspect from left
+            # range(0, 128) means 0-127
+            for col in range(0, mid_range):
+                t_idx = bundle.curr_index[128 * row + col]
+                t_size = int(math.floor(t_idx / Bundle.M))
+                if data_started:
+                    if t_size == 0:
+                        absrow = bundle.row_offset + row
+                        abscol = bundle.col_offset + col
+                        files.append(dict(col=abscol, row=absrow, lvl=int(bundle.level)))
+                else:
+                    if t_size != 0:
+                        data_started = True
+            data_started = False
+            # inspect from right
+            for col in range(127, mid_range - 1, -1):
+                t_idx = bundle.curr_index[128 * row + col]
+                t_size = int(math.floor(t_idx / Bundle.M))
+                if data_started:
+                    if t_size == 0:
+                        absrow = bundle.row_offset + row
+                        abscol = bundle.col_offset + col
+                        files.append(dict(col=abscol, row=absrow, lvl=int(bundle.level)))
+                else:
+                    if t_size != 0:
+                        data_started = True
+
+    return files
+
+if __name__ == '__main__':
+    main()